Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of henry.saputra@gmail.com
 designates 209.85.223.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <2AC0EE0D-7F45-4976-BD0A-8E10EAE97225@toolazydogs.com>
References: 
 <CAOv+hsSvAgL0d=EY7i4UfLf+JNHp8N=a0TrDrfrLPCxEvGnKvg@mail.gmail.com>
	<6710E889-FC6A-4149-A6EB-935D9463038C@hortonworks.com>
	<CAJwFCa0fPOy+0mzgF9VwjA2MPwi7wxryDvwp0gJWiC8nK4qWcw@mail.gmail.com>
	<2AC0EE0D-7F45-4976-BD0A-8E10EAE97225@toolazydogs.com>
Date: Wed, 4 Sep 2013 11:47:14 -0700
Message-ID: 
 <CALuGr6a_qVYcDRUz4rU1YYdBD2oGiZ1x450MZ_sFXYqMeMU5Hg@mail.gmail.com>
Subject: Re: [PROPOSAL] Storm for Apache Incubator
From: Henry Saputra <henry.saputra@gmail.com>
To: "general@incubator.apache.org" <general@incubator.apache.org>
Content-Type: multipart/alternative; boundary=001a11c3b834e65e2c04e5933bd7

--001a11c3b834e65e2c04e5933bd7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I think this is just PROPOSAL thread. Some questions asked in this thread
so I guess no VOTE yet.

- Henry


On Wed, Sep 4, 2013 at 10:14 AM, Alan D. Cabrera <list@toolazydogs.com>wrot=
e:

> Are we voting?
>
> Regards,
> Alan
>
> On Sep 4, 2013, at 8:33 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
>
> > +1 binding
> >
> >
> > On Wed, Sep 4, 2013 at 7:52 AM, Suresh Srinivas <suresh@hortonworks.com
> >wrote:
> >
> >> +1 (non-binding)
> >>
> >> Sent from phone
> >>
> >> On Sep 4, 2013, at 1:07 AM, Nathan Marz <nathan@nathanmarz.com> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> I'd like to propose Storm to be an Apache Incubator project. After mu=
ch
> >>> thought I believe this is the right next step for the project, and I
> look
> >>> forward to hearing everyone's thoughts and feedback!
> >>>
> >>> Here's a link to the proposal:
> >>> https://wiki.apache.org/incubator/StormProposal
> >>>
> >>> The proposal is also pasted below.
> >>>
> >>> -Nathan
> >>>
> >>>
> >>> =3D Storm Proposal =3D
> >>>
> >>> =3D=3D Abstract =3D=3D
> >>>
> >>> Storm is a distributed, fault-tolerant, and high-performance realtime
> >>> computation system that provides strong guarantees on the processing =
of
> >>> data.
> >>>
> >>> =3D=3D Proposal =3D=3D
> >>>
> >>> Storm is a distributed real-time computation system. Similar to how
> >> Hadoop
> >>> provides a set of general primitives for doing batch processing, Stor=
m
> >>> provides a set of general primitives for doing real-time computation.
> Its
> >>> use cases span stream processing, distributed RPC, continuous
> >> computation,
> >>> and more. Storm has become a preferred technology for near-realtime
> >>> big-data processing by many organizations worldwide (see a partial li=
st
> >> at
> >>> https://github.com/nathanmarz/storm/wiki/Powered-By). As an open
> source
> >>> project, Storm=E2=80=99s developer community has grown rapidly to 46 =
members.
> >>>
> >>> =3D=3D Background =3D=3D
> >>>
> >>> The past decade has seen a revolution in data processing. MapReduce,
> >>> Hadoop, and related technologies have made it possible to store and
> >> process
> >>> data at scales previously unthinkable. Unfortunately, these data
> >> processing
> >>> technologies are not realtime systems, nor are they meant to be. The
> lack
> >>> of a "Hadoop of realtime" has become the biggest hole in the data
> >>> processing ecosystem. Storm fills that hole.
> >>>
> >>> Storm was initially developed and deployed at BackType in 2011. After=
 7
> >>> months of development BackType was acquired by Twitter in July 2011.
> >> Storm
> >>> was open sourced in September 2011.
> >>>
> >>> Storm has been under continuous development on its Github repository
> >> since
> >>> being open-sourced. It has undergone four major releases (0.5, 0.6,
> 0.7,
> >>> 0.8) and many minor ones.
> >>>
> >>> =3D=3D Rationale =3D=3D
> >>>
> >>> Storm is a general platform for low-latency big-data processing. It i=
s
> >>> complementary to the existing Apache projects, such as Hadoop. Many
> >>> applications are actually exploring using both Hadoop and Storm for
> >>> big-data processing. Bringing Storm into Apache is very beneficial to
> >> both
> >>> Apache community and Storm community.
> >>>
> >>> The rapid growth of Storm community is empowered by open source. We
> >> believe
> >>> the Apache foundation is a great fit as the long-term home for Storm,
> as
> >> it
> >>> provides an established process for community-driven development and
> >>> decision making by consensus. This is exactly the model we want for
> >> future
> >>> Storm development.
> >>>
> >>> =3D=3D Initial Goals =3D=3D
> >>>
> >>> * Move the existing codebase to Apache
> >>> * Integrate with the Apache development process
> >>> * Ensure all dependencies are compliant with Apache License version 2=
.0
> >>> * Incremental development and releases per Apache guidelines
> >>>
> >>> =3D=3D Current Status =3D=3D
> >>>
> >>> Storm has undergone four major releases (0.5, 0.6, 0.7, 0.8) and many
> >> minor
> >>> ones. Storm 0.9 is about to be released. Storm is being used in
> >> production
> >>> by over 50 organizations. Storm codebase is currently hosted at
> >> github.com,
> >>> which will seed the Apache git repository.
> >>>
> >>> =3D=3D=3D Meritocracy =3D=3D=3D
> >>>
> >>> We plan to invest in supporting a meritocracy. We will discuss the
> >>> requirements in an open forum. Several companies have already express=
ed
> >>> interest in this project, and we intend to invite additional develope=
rs
> >> to
> >>> participate. We will encourage and monitor community participation so
> >> that
> >>> privileges can be extended to those that contribute.
> >>>
> >>> =3D=3D=3D Community =3D=3D=3D
> >>>
> >>> The need for a low-latency big-data processing platform in the open
> >> source
> >>> is tremendous. Storm is currently being used by at least 50
> organizations
> >>> worldwide (see https://github.com/nathanmarz/storm/wiki/Powered-By),
> >> and is
> >>> the most starred Java project on Github. By bringing Storm into Apach=
e,
> >> we
> >>> believe that the community will grow even bigger.
> >>>
> >>> =3D=3D=3D Core Developers =3D=3D=3D
> >>>
> >>> Storm was started by Nathan Marz at BackType, and now has developers
> from
> >>> Yahoo!, Microsoft, Alibaba, Infochimps, and many other companies.
> >>>
> >>> =3D=3D=3D Alignment =3D=3D=3D
> >>>
> >>> In the big-data processing ecosystem, Storm is a very popular
> low-latency
> >>> platform, while Hadoop is the primary platform for batch processing. =
We
> >>> believe that it will help the further growth of big-data community by
> >>> having Hadoop and Storm aligned within Apache foundation. The alignme=
nt
> >> is
> >>> also beneficial to other Apache communities (such as Zookeeper, Thrif=
t,
> >>> Mesos). We could include additional sub-projects, Storm-on-YARN and
> >>> Storm-on-Mesos, in the near future.
> >>>
> >>> =3D=3D Known Risks =3D=3D
> >>>
> >>> =3D=3D=3D Orphaned Products =3D=3D=3D
> >>>
> >>> The risk of the Storm project being abandoned is minimal. There are a=
t
> >>> least 50 organizations (Twitter, Yahoo!, Microsoft, Groupon, Baidu,
> >>> Alibaba, Alipay, Taobao, PARC, RocketFuel etc) are highly incentivize=
d
> to
> >>> continue development. Many of these organizations have built critical
> >>> business applications upon Storm, and have devoted significant intern=
al
> >>> infrastructure investment in Storm.
> >>>
> >>> =3D=3D=3D Inexperience with Open Source =3D=3D=3D
> >>>
> >>> Storm has existed as a healthy open source project for several years.
> >>> During that time, we have curated an open-source community
> successfully,
> >>> attracting over 40 developers from a diverse group of companies
> including
> >>> Twitter, Yahoo!, and Alibaba.
> >>>
> >>> =3D=3D=3D Homogenous Developers =3D=3D=3D
> >>>
> >>> The initial committers are employed by large companies (including
> >> Twitter,
> >>> Yahoo!, Alibaba, Microsoft) and well-funded startups. Storm has an
> active
> >>> community of developers, and we are committed to recruiting additiona=
l
> >>> committers based on their contributions to the project.
> >>>
> >>> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
> >>>
> >>> It is expected that Storm development will occur on both salaried tim=
e
> >> and
> >>> on volunteer time, after hours. The majority of initial committers ar=
e
> >> paid
> >>> by their employer to contribute to this project. However, they are al=
l
> >>> passionate about the project, and we are confident that the project
> will
> >>> continue even if no salaried developers contribute to the project. We
> are
> >>> committed to recruiting additional committers including non-salaried
> >>> developers.
> >>>
> >>> =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D
> >>>
> >>> As mentioned in the Alignment section, Storm is closely integrated wi=
th
> >>> Hadoop,
> >>> Zookeeper, Thrift, YARN and Mesos in a numerous ways. We look forward
> to
> >>> collaborating with those communities, as well as other Apache
> communities
> >>> (including Apache S4 which focuses on stateful low-latency processing=
).
> >>>
> >>> =3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D
> >>>
> >>> Storm is already a healthy and well known open source project. This
> >>> proposal is not for the purpose of generating publicity. Rather, the
> >>> primary benefits to joining Apache are those outlined in the Rational=
e
> >>> section.
> >>>
> >>> =3D=3D Documentation =3D=3D
> >>>
> >>> The reader will find these websites highly relevant:
> >>>
> >>> * Storm website: http://storm-project.net
> >>> * Storm documentation: https://github.com/nathanmarz/storm/wiki
> >>> * Codebase: https://github.com/nathanmarz/storm
> >>> * User group: https://groups.google.com/group/storm-user
> >>>
> >>> =3D=3D Source and Intellectual Property Submission Plan =3D=3D
> >>>
> >>> The Storm codebase is currently hosted on Github:
> >>> https://github.com/nathanmarz/storm.
> >>>
> >>> This is the exact codebase that we would migrate to the Apache
> >> foundation.
> >>>
> >>> The Storm source code is currently licensed under Eclipse Public
> License
> >>> Version 1.0. Some source code was contributed under a contributor
> >> agreement
> >>> based on the Sun contributor agreement (v1.5). More recent code has
> been
> >>> contributed under an Apache style agreement (see
> >>>
> https://dl.dropboxusercontent.com/u/133901206/storm-apache-style-cla.txt
> >> ).
> >>>
> >>> Upon entering Apache, Storm will migrate to an Apache License 2.0 wit=
h
> >> all
> >>> contributions licensed to the Apache Foundation. In certain cases whe=
re
> >>> individuals or organizations hold copyright, we will ensure they gran=
t
> a
> >>> license to the Apache Foundation. Going forward, all commits will be
> >>> licensed directly to the Apache foundation through our signed
> Individual
> >>> Contributor License Agreements for all committers on the project.
> >>>
> >>> Yahoo! is also willing to move Storm-on-YARN code from github to be a
> >>> subproject of Apache Storm project. Storm-on-YARN is currently licens=
ed
> >>> under Apache License 2.0 and receive contribution under Apache style
> CLA.
> >>> Upon entering Apache, Yahoo! will sign over copyright to Apache
> >> foundation.
> >>>
> >>> =3D=3D External Dependencies =3D=3D
> >>>
> >>> To the best of our knowledge, all of Storm dependencies (except
> 0MQ/JMQ)
> >>> are distributed under Apache compatible licenses. Upon acceptance to
> the
> >>> incubator, we would begin a thorough analysis of all transitive
> >>> dependencies to verify this fact and introduce license checking into
> the
> >>> build and release process (for instance integrating Apache Rat).
> >>>
> >>> Storm has used 0MQ and JMQ as the default mechanism for internal
> >> messaging
> >>> layer, and 0MQ/JMQ is licensed under GNU Lesser General Public Licens=
e.
> >>> Recently, we have made Storm messaging layer pluggable, and plan to u=
se
> >>> Netty (which is licensed under Apache License v2) as our default
> >> messaging
> >>> plugin (while keep 0MQ as an optional plugin).
> >>>
> >>> =3D=3D Cryptography =3D=3D
> >>>
> >>> We do not expect Storm to be a controlled export item due to the use =
of
> >>> encryption.
> >>>
> >>> Storm enable encryptions via 2 plugins:
> >>>
> >>> * SASL authentication plugins =E2=80=A6 Currently, we have provide =
=E2=80=9Cno-op=E2=80=9D
> >>> authentication and digest authentication. In near future, we will
> >> introduce
> >>> Kerberos authentication.
> >>> * Tuple payload serialization plugins =E2=80=A6 Storm provides plugin=
s for
> >>> plain-object serialization and blowfish encryption.
> >>>
> >>> =3D=3D Required Resources =3D=3D
> >>>
> >>> =3D=3D=3D Mailing lists =3D=3D=3D
> >>>
> >>> * storm-user
> >>> * storm-dev
> >>> * storm-private (with moderated subscriptions)
> >>>
> >>> =3D=3D=3D Subversion Directory =3D=3D=3D
> >>>
> >>> Git is the preferred source control system: git://git.apache.org/stor=
m
> >>>
> >>> =3D=3D=3D Issue Tracking =3D=3D=3D
> >>>
> >>> JIRA Storm (STORM)
> >>>
> >>> =3D=3D Initial Committers =3D=3D
> >>>
> >>> * Nathan Marz <nathan at nathanmarz dot com>
> >>> * James Xu <xumingmingv at gmail dot com>
> >>> * Jason Jackson <jason at cvk dot ca>
> >>> * Andy Feng <afeng at yahoo-inc dot com>
> >>> * Flip Kromer  <flip at infochimps dot com>
> >>> * David Lao <davidlao at microsoft dot com>
> >>> * P. Taylor Goetz <ptgoetz at gmail dot com>
> >>>
> >>> =3D=3D Affiliations =3D=3D
> >>>
> >>> * Nathan Marz - Nathan=E2=80=99s Startup
> >>> * James Xu - Alibaba
> >>> * Jason Jackson - Twitter
> >>> * Andy Feng - Yahoo!
> >>> * Flip Kromer - Infochimps
> >>> * David Lao - Microsoft
> >>> * P. Taylor Goetz - Health Market Science
> >>>
> >>> =3D=3D Sponsors =3D=3D
> >>>
> >>> =3D=3D=3D Champion =3D=3D=3D
> >>>
> >>> * Doug Cutting  <cutting at apache dot org>
> >>>
> >>> =3D=3D=3D Nominated Mentors =3D=3D=3D
> >>>
> >>> * Ted Dunning <tdunning at maprtech.com>
> >>> * Arvind Prabhaker <arvind at apache dot org>
> >>> * Devaraj Das <ddas at hortonworks dot com>
> >>>
> >>> =3D=3D=3D Sponsoring Entity =3D=3D=3D
> >>>
> >>> The Apache Incubator
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential=
,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: general-help@incubator.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

--001a11c3b834e65e2c04e5933bd7--