incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Jagielski <...@jaguNET.com>
Subject Re: [VOTE] Pulsar into the Apache Incubator
Date Wed, 17 May 2017 14:15:54 GMT
Make it so! +1! (binding)

> On May 16, 2017, at 10:39 PM, Bryan Call <bcall@apache.org> wrote:
> 
> Hi All,
> 
> As the champion for Pulsar, I would like to start a VOTE to bring the
> project in as Apache incubator podling.
> 
> The ASF voting rules are described:
> 
> 	https://www.apache.org/foundation/voting.html
> 
> A vote for accepting a new Apache Incubator podling is a majority vote for which 
> only Incubator PMC member votes are binding.
> 
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Pulsar into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Pulsar into the Apache Incubator because ...
> 
> The proposal is listed below, but you can also access it on the wiki:
> 
> 	https://wiki.apache.org/incubator/PulsarProposal
> 
> -Bryan
> 
> 
> = Pulsar Proposal =
> 
> == Abstract ==
> 
> Pulsar is a highly scalable, low latency messaging platform running on
> commodity hardware. It provides simple pub-sub semantics over topics,
> guaranteed at-least-once delivery of messages, automatic cursor management for
> subscribers, and cross-datacenter replication.
> 
> == Proposal ==
> 
> Pub-sub messaging is a very common design pattern that is increasingly found
> in distributed systems powering Internet applications. These applications
> provide real-time services, and need publish-latencies of 5ms on average and
> no more than 15ms at the 99th percentile. At Internet scale, these
> applications require a messaging system with ordering, strong durability, and
> delivery guarantees. In order to handle the “five 9’s” durability requirements
> of a production environment, the messages have to be committed on multiple
> disks or nodes.
> 
> Pulsar has been developed at Yahoo to address these specific requirements by
> providing a hosted service supporting millions of topics for multiple tenants.
> The current incarnation of Pulsar has been open-sourced under Apache license
> in September 2016 and it is the direct evolution of systems that were
> developed at Yahoo since 2011.
> 
> We believe there is currently no other system that provides a multi-tenant
> hosted messaging platform capable of supporting a huge number of topics while
> maintaining strict guarantees for durability, ordering and low latency.
> Current solutions would require to run multiple individual clusters with
> additional operational work and capacity overhead.
> 
> Since the open sourcing of Pulsar, the development has been done exclusively
> on the public Github repository and two major releases were shipped (1.15 and
> 1.16), along with multiple minor ones. Several other companies have expressed
> interest in the project and its future direction.
> 
> == Rationale ==
> 
> Pulsar is a platform that is built on top of several other Apache projects. In
> particular, Apache BookKeeper is used to store the data and Apache ZooKeeper
> is used for coordination and metadata storage. Pulsar is also interoperable
> out of the box with Apache Storm, to provide an easy to use stream processing
> solution.
> 
> We want to establish a community outside the scope of initial core developers
> at Yahoo and we believe that the Apache Foundation is a great fit and long-
> term home for Pulsar, as it provides an established process for community-
> driven development and decision making by consensus. This is exactly the model
> we want to adopt for future Pulsar development.
> 
> == Initial Goals ==
> 
> The initial goals will be to move the existing codebase to Apache and
> integrate with the Apache development process. Furthermore, we plan for
> incremental development, and releases along with the Apache guidelines.
> 
> == Current Status ==
> 
> Pulsar has been in service at large scale for more than 2 years at Yahoo. In
> this time around 60 different applications were integrated with Pulsar. Other
> companies are evaluating it as well and have been contributing code to the
> project.
> 
> === Meritocracy ===
> 
> We value meritocracy and we understand that it is the basis to form an open
> community that encourages multiple companies and individuals to contribute and
> get invested in the project future. We will encourage and monitor
> participation and make sure to extend privileges and responsibilities to all
> contributors.
> 
> === Community ===
> 
> We have validated, through the interest demonstrated by Pulsar users at Yahoo,
> that a reliable hosted pub-sub messaging platform represent a very important
> building block for web-scale distributed applications. We believe that many
> companies can benefit by applying the same model and that bringing Pulsar to
> Apache will get the community to grow stronger.
> 
> === Core Developers ===
> 
> Pulsar has been initially developed at Yahoo and received significant
> contributions from Yahoo Japan. After having open-sourced the project there
> have been contribution from developers from several external companies.
> 
> === Alignment ===
> 
> Pulsar builds upon other Apache projects such as ZooKeeper and BookKeeper,
> along with a number of other Apache libraries. We have already integrated with
> Storm and we envision to integrate with multiple other systems in the
> streaming and big data space.
> 
> == Known Risks ==
> 
> === Orphaned Products ===
> 
> Yahoo has been doing most of the development and, given that many internal
> platforms depends on Pulsar, it is heavily invested in the long term success
> of the the project. Yahoo has a long history participating in open-source
> projects, and has been also a long time contributor to the Apache community.
> 
> === Inexperience with Open Source ===
> 
> Many Pulsar contributors are already familiar with the open source process and
> several of them are committers on other Apache projects. We will be actively
> working with experienced Apache community members to improve our project.
> 
> === Homogenous Developers ===
> 
> The initial committers are employed by large companies including Yahoo, Yahoo!
> Japan, Salesforce and MercadoLibre. We hope to grow the community and to
> include additional committers based on their contributions to the project.
> 
> === Reliance on Salaried Developers ===
> 
> It is expected that Pulsar development will occur on both salaried time and on
> volunteer time, after hours. The majority of initial committers are paid by
> their employer to contribute to this project. However, they are all passionate
> about the project, and we are confident that the project will continue even if
> no salaried developers contribute to the project.
> 
> === Relationships with Other Apache Products ===
> 
> As mentioned in the Rationale section, Pulsar is closely dependent and
> integrated with BookKeeper and ZooKeeper and Storm. There are ongoing to
> integrate with other projects such Apache Spark. We look forward to
> collaborating with those communities, as well as other Apache communities.
> 
> === An Excessive Fascination with the Apache Brand ===
> 
> We are applying to the Incubator process because we think it is the next
> logical step for the Pulsar project after open-sourcing the code in 2016. This
> proposal is not for the purpose of generating publicity. Rather, we want to
> make sure to create a very inclusive and meritocratic community, outside the
> umbrella of a single company. Yahoo has a long standing history of
> contributing to Apache projects and the Pulsar developers and contributors
> understand the implication of making it an Apache project.
> 
> == Documentation ==
> * Pulsar code base: https://github.com/yahoo/pulsar
> * Pulsar documentation: https://github.com/yahoo/pulsar/blob
> 	/master/docs/Documentation.md
> * Blog post:  [[https://yahooeng.tumblr.com/post/150078336821/
> 	open-sourcing-pulsar-pub-sub-messaging-at-scale|Open-sourcing Pulsar,
> 	Pub-sub Messaging at Scale]]
> 
> == Initial Source ==
> 
> The Pulsar codebase is currently hosted on Github:
> https://github.com/yahoo/pulsar. This is the exact codebase that we would
> migrate to the Apache Software Foundation.
> 
> == Source and Intellectual Property Submission Plan ==
> 
> The Pulsar source code in Github is currently licensed under Apache License
> v2.0 and the copyright is assigned to Yahoo. All the contributions from
> external parties have been received under Apache style CLA. If Pulsar fulfills
> and passes the conditions for being an Incubator project in the ASF, Yahoo
> will transition the source code ownership to the Apache Software Foundation
> via the Software Grant Agreement.
> 
> == External Dependencies ==
> 
> To the best of our knowledge, all of Pulsar dependencies are distributed under
> Apache compatible licenses.
> 
> === External dependencies licensed under Apache License 2.0: ===
> 
> Athenz, JCommander, HPPC - High Performance Primitive Collections for Java,
> FasterXML Jackson, Caffeine Async Cache, GSon, Guava, Netty, DataSketches,
> Joda-time, Jna Java Native Access, Lz4-java, AsyncHttpClient, Jetty, SnakeYAML
> 
> === ASF Projects: ===
> 
> BookKeeper, ZooKeeper, Storm, Log4J, Commons (BeanUtils, CLI,  Codec,
> Collections, Configuration, Digester, IO, Lang, Lang3, Logging)
> 
> === Others: ===
> * Protobuf (3-clause BSD)
> * JLine (BSD License)
> * Jersey (CDDL - Version 1.1)
> * HdrHistogram (BSD License)
> * RocksDB-JNI (3-clause BSD)
> * SLF4J API (MIT)
> 
> == Required Resources ==
> 
> === Mailing lists ===
> * users@pulsar.incubator.apache.org
> * dev@pulsar.incubator.apache.org
> * commits@pulsar.incubator.apache.org
> * private@pulsar.incubator.apache.org (with moderated subscriptions)
> 
> === Git Repository ===
> * https://git-wip-us.apache.org/repos/asf/incubator-pulsar
> 
> === Issue Tracking ===
> * JIRA Pulsar (PULSAR)
> 
> == Initial Committers ==
> * Matteo Merli - <mmerli@apache.org>
> * Joe Francis - <joef@yahoo-inc.com>
> * Rajan Dhabalia - <rdhabalia@yahoo-inc.com>
> * Sahaya Andrews Albert - <sandrews@yahoo-inc.com>
> * Maurice Barnum - <msb@yahoo-inc.com>
> * Ludwig Pummer - <ludwig@yahoo-inc.com>
> * Jai Asher - <jai1@yahoo-inc.com>
> * Siddharth Boobna - <sboobna@apache.org>
> * Nozomi Kurihara - <nkurihar@yahoo-corp.jp>
> * Yuki Shiga - <yushiga@yahoo-corp.jp>
> * Masakazu Kitajo - <maskit@apache.org>
> * Sebastián Schepens - <sebastian.schepens@mercadolibre.com>
> * Brad McMillen - <bradtm@yahoo-inc.com>
> * Bobbey Reese - <breese@yahoo-inc.com>
> * Masahiro Sakamoto <massakam@yahoo-corp.jp>
> * Hiroyuki Sakai <hsakai@yahoo-corp.jp>
> 
> == Affiliations ==
> * Matteo Merli - Streamlio
> * Joe Francis - Yahoo
> * Rajan Dhabalia - Yahoo
> * Sahaya Andrews Albert - Yahoo
> * Maurice Barnum - Yahoo
> * Ludwig Pummer - Yahoo
> * Jai Asher - Yahoo
> * Siddharth Boobna - Salesforce
> * Nozomi Kurihara - Yahoo! Japan
> * Yuki Shiga - Yahoo! Japan
> * Masakazu Kitajo - Apple
> * Sebastián Schepens - Mercado Libre
> * Brad McMillen - Yahoo
> * Bobbey Reese - Yahoo
> 
> == Sponsors ==
> 
> === Champion ===
> * Bryan Call
> 
> === Nominated Mentors ===
> * Dave Fisher
> * Jim Jagielski
> * P. Taylor Goetz
> * Francis Liu
> 
> === Sponsoring Entity ===
> * The Apache Incubator PMC   
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message