Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
MIME-Version: 1.0
Reply-To: sharad@apache.org
In-Reply-To: 
 <CALEq1Z8sL0uGdEpFKMwjsS0vc1aw33ekrMWfRmbP6Ktnjy0REg@mail.gmail.com>
References: 
 <CALEq1Z8sL0uGdEpFKMwjsS0vc1aw33ekrMWfRmbP6Ktnjy0REg@mail.gmail.com>
Date: Sat, 14 Sep 2013 18:33:26 +0530
Message-ID: 
 <CAPqfsdw3FsfPr4NoknDTc_TyOK898jq5cQgQzxhoeSmRYE4=2Q@mail.gmail.com>
Subject: Re: [VOTE] Accept Storm into the Incubator
From: Sharad Agarwal <sharad@apache.org>
To: general@incubator.apache.org
Content-Type: multipart/alternative; boundary=001a11c3e47cce7f7604e6579829

--001a11c3e47cce7f7604e6579829
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

+1 (non-binding)


On Fri, Sep 13, 2013 at 12:49 AM, Doug Cutting <cutting@apache.org> wrote:

> Discussion about the Storm proposal has subsided, issues raised now
> seemingly resolved.
>
> I'd like to call a vote to accept Storm as a new Incubator podling.
>
> The proposal is included below and is also at:
>
>   https://wiki.apache.org/incubator/StormProposal
>
> Let's keep the vote open for four working days, until 18 September.
>
> [ ] +1 Accept Storm into the Incubator
> [ ] +0 Don't care.
> [ ] -1 Don't accept Storm because...
>
> Doug
>
>
> =3D Storm Proposal =3D
>
> =3D=3D Abstract =3D=3D
>
> Storm is a distributed, fault-tolerant, and high-performance realtime
> computation system that provides strong guarantees on the processing
> of data.
>
> =3D=3D Proposal =3D=3D
>
> Storm is a distributed real-time computation system. Similar to how
> Hadoop provides a set of general primitives for doing batch
> processing, Storm provides a set of general primitives for doing
> real-time computation. Its use cases span stream processing,
> distributed RPC, continuous computation, and more. Storm has become a
> preferred technology for near-realtime big-data processing by many
> organizations worldwide (see a partial list at
> https://github.com/nathanmarz/storm/wiki/Powered-By). As an open
> source project, Storm=92s developer community has grown rapidly to 46
> members.
>
> =3D=3D Background =3D=3D
>
> The past decade has seen a revolution in data processing. MapReduce,
> Hadoop, and related technologies have made it possible to store and
> process data at scales previously unthinkable. Unfortunately, these
> data processing technologies are not realtime systems, nor are they
> meant to be. The lack of a "Hadoop of realtime" has become the biggest
> hole in the data processing ecosystem. Storm fills that hole.
>
> Storm was initially developed and deployed at BackType in 2011. After
> 7 months of development BackType was acquired by Twitter in July 2011.
> Storm was open sourced in September 2011.
>
> Storm has been under continuous development on its Github repository
> since being open-sourced. It has undergone four major releases (0.5,
> 0.6, 0.7, 0.8) and many minor ones.
>
>
> =3D=3D Rationale =3D=3D
>
> Storm is a general platform for low-latency big-data processing. It is
> complementary to the existing Apache projects, such as Hadoop. Many
> applications are actually exploring using both Hadoop and Storm for
> big-data processing. Bringing Storm into Apache is very beneficial to
> both Apache community and Storm community.
>
> The rapid growth of Storm community is empowered by open source. We
> believe the Apache foundation is a great fit as the long-term home for
> Storm, as it provides an established process for community-driven
> development and decision making by consensus. This is exactly the
> model we want for future Storm development.
>
> =3D=3D Initial Goals =3D=3D
>
>    * Move the existing codebase to Apache
>    * Integrate with the Apache development process
>    * Ensure all dependencies are compliant with Apache License version 2.=
0
>    * Incremental development and releases per Apache guidelines
>
> =3D=3D Current Status =3D=3D
>
> Storm has undergone four major releases (0.5, 0.6, 0.7, 0.8) and many
> minor ones. Storm 0.9 is about to be released. Storm is being used in
> production by over 50 organizations. Storm codebase is currently
> hosted at github.com, which will seed the Apache git repository.
>
> =3D=3D=3D Meritocracy =3D=3D=3D
>
> We plan to invest in supporting a meritocracy. We will discuss the
> requirements in an open forum. Several companies have already
> expressed interest in this project, and we intend to invite additional
> developers to participate. We will encourage and monitor community
> participation so that privileges can be extended to those that
> contribute.
>
> =3D=3D=3D Community =3D=3D=3D
>
> The need for a low-latency big-data processing platform in the open
> source is tremendous. Storm is currently being used by at least 50
> organizations worldwide (see
> https://github.com/nathanmarz/storm/wiki/Powered-By), and is the most
> starred Java project on Github. By bringing Storm into Apache, we
> believe that the community will grow even bigger.
>
> =3D=3D=3D Core Developers =3D=3D=3D
>
> Storm was started by Nathan Marz at BackType, and now has developers
> from Yahoo!, Microsoft, Alibaba, Infochimps, and many other companies.
>
> =3D=3D=3D Alignment =3D=3D=3D
>
> In the big-data processing ecosystem, Storm is a very popular
> low-latency platform, while Hadoop is the primary platform for batch
> processing. We believe that it will help the further growth of
> big-data community by having Hadoop and Storm aligned within Apache
> foundation. The alignment is also beneficial to other Apache
> communities (such as Zookeeper, Thrift, Mesos). We could include
> additional sub-projects, Storm-on-YARN and Storm-on-Mesos, in the near
> future.
>
> =3D=3D Known Risks =3D=3D
>
> =3D=3D=3D Orphaned Products =3D=3D=3D
>
> The risk of the Storm project being abandoned is minimal. There are at
> least 50 organizations (Twitter, Yahoo!, Microsoft, Groupon, Baidu,
> Alibaba, Alipay, Taobao, PARC, RocketFuel etc) are highly incentivized
> to continue development. Many of these organizations have built
> critical business applications upon Storm, and have devoted
> significant internal infrastructure investment in Storm.
>
> =3D=3D=3D Inexperience with Open Source =3D=3D=3D
>
> Storm has existed as a healthy open source project for several years.
> During that time, we have curated an open-source community
> successfully, attracting over 40 developers from a diverse group of
> companies including Twitter, Yahoo!, and Alibaba.
>
> =3D=3D=3D Homogenous Developers =3D=3D=3D
>
> The initial committers are employed by large companies (including
> Twitter, Yahoo!, Alibaba, Microsoft) and well-funded startups. Storm
> has an active community of developers, and we are committed to
> recruiting additional committers based on their contributions to the
> project.
>
> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
>
> It is expected that Storm development will occur on both salaried time
> and on volunteer time, after hours. The majority of initial committers
> are paid by their employer to contribute to this project. However,
> they are all passionate about the project, and we are confident that
> the project will continue even if no salaried developers contribute to
> the project. We are committed to recruiting additional committers
> including non-salaried developers.
>
> =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D
>
> As mentioned in the Alignment section, Storm is closely integrated with
> Hadoop,
> Zookeeper, Thrift, YARN and Mesos in a numerous ways. We look forward
> to collaborating with those communities, as well as other Apache
> communities (including Apache S4 which focuses on stateful low-latency
> processing).
>
> =3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D
>
> Storm is already a healthy and well known open source project. This
> proposal is not for the purpose of generating publicity. Rather, the
> primary benefits to joining Apache are those outlined in the Rationale
> section.
>
> =3D=3D Documentation =3D=3D
>
> The reader will find these websites highly relevant:
>    * Storm website: http://storm-project.net
>    * Storm documentation: https://github.com/nathanmarz/storm/wiki
>    * Codebase: https://github.com/nathanmarz/storm
>    * User group: https://groups.google.com/group/storm-user
>
> =3D=3D Source and Intellectual Property Submission Plan =3D=3D
>
> The Storm codebase is currently hosted on Github:
> https://github.com/nathanmarz/storm.
> This is the exact codebase that we would migrate to the Apache foundation=
.
>
> The Storm source code is currently licensed under Eclipse Public
> License Version 1.0. Some source code was contributed under a
> contributor agreement based on the Sun contributor agreement (v1.5).
> More recent code has been contributed under an Apache style agreement
> (see
> https://dl.dropboxusercontent.com/u/133901206/storm-apache-style-cla.txt)=
.
>
> Upon entering Apache, Storm will migrate to an Apache License 2.0 with
> all contributions licensed to the Apache Foundation. In certain cases
> where individuals or organizations hold copyright, we will ensure they
> grant a license to the Apache Foundation. Going forward, all commits
> will be licensed directly to the Apache foundation through our signed
> Individual Contributor License Agreements for all committers on the
> project.
>
> storm-kafka, which lets one use Kafka as a source for Storm, will also
> be submitted under the contrib folder for the Apache Storm project.
>
> Yahoo! is also willing to move Storm-on-YARN code from github to be a
> subproject of Apache Storm project. Storm-on-YARN is currently
> licensed under Apache License 2.0 and receive contribution under
> Apache style CLA. Upon entering Apache, Yahoo! will sign over
> copyright to Apache foundation.
>
> =3D=3D External Dependencies =3D=3D
>
> To the best of our knowledge, all of Storm dependencies (except
> 0MQ/JMQ) are distributed under Apache compatible licenses. Upon
> acceptance to the incubator, we would begin a thorough analysis of all
> transitive dependencies to verify this fact and introduce license
> checking into the build and release process (for instance integrating
> Apache Rat).
>
> Storm has used 0MQ and JMQ as the default mechanism for internal
> messaging layer, and 0MQ/JMQ is licensed under GNU Lesser General
> Public License. Recently, we have made Storm messaging layer
> pluggable, and plan to use Netty (which is licensed under Apache
> License v2) as our default messaging plugin (while keep 0MQ as an
> optional plugin).
>
> =3D=3D Cryptography =3D=3D
>
> We do not expect Storm to be a controlled export item due to the use
> of encryption.
> Storm enable encryptions via 2 plugins:
>    * SASL authentication plugins =85 Currently, we have provide =93no-op=
=94
> authentication and digest authentication. In near future, we will
> introduce Kerberos authentication.
>    * Tuple payload serialization plugins =85 Storm provides plugins for
> plain-object serialization and blowfish encryption.
>
> =3D=3D Required Resources =3D=3D
>
> =3D=3D=3D Mailing lists =3D=3D=3D
>
>  * storm-user
>  * storm-dev
>  * storm-commits
>  * storm-private (with moderated subscriptions)
>
> =3D=3D=3D Subversion Directory =3D=3D=3D
>
> Git is the preferred source control system: git://git.apache.org/storm
>
>
> =3D=3D=3D Issue Tracking =3D=3D=3D
>
> JIRA Storm (STORM)
>
> =3D=3D Initial Committers =3D=3D
>
>    * Nathan Marz <nathan at nathanmarz dot com>
>    * James Xu <xumingmingv at gmail dot com>
>    * Jason Jackson <jason at cvk dot ca>
>    * Andy Feng <afeng at yahoo-inc dot com>
>    * Flip Kromer  <flip at infochimps dot com>
>    * David Lao <davidlao at microsoft dot com>
>    * P. Taylor Goetz <ptgoetz at gmail dot com>
>
> =3D=3D Affiliations =3D=3D
>
>    * Nathan Marz - Nathan=92s Startup
>    * James Xu - Alibaba
>    * Jason Jackson - Twitter
>    * Andy Feng - Yahoo!
>    * Flip Kromer - Infochimps
>    * David Lao - Microsoft
>    * P. Taylor Goetz - Health Market Science
>
> =3D=3D Sponsors =3D=3D
>
>
> =3D=3D=3D Champion =3D=3D=3D
>
>    * Doug Cutting  <cutting at apache dot org>
>
> =3D=3D=3D Nominated Mentors =3D=3D=3D
>
>   * Ted Dunning <tdunning at maprtech dot com>
>   * Arvind Prabhakar <arvind at apache dot org>
>   * Devaraj Das <ddas at hortonworks dot com>
>   * Matt Franklin <m.ben.franklin at gmail dot com>
>   * Benjamin Hindman <benjamin.hindman at gmail dot com>
>
> =3D=3D=3D Sponsoring Entity =3D=3D=3D
>
>  The Apache Incubator
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

--001a11c3e47cce7f7604e6579829--