Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of gates@hortonworks.com
 designates 209.85.220.49 as permitted sender)
Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\))
Subject: [RESULT] [VOTE] Accept Stratosphere into the incubator
From: Alan Gates <gates@hortonworks.com>
In-Reply-To: <29230441-F926-4A2C-BE98-A5D29C813897@hortonworks.com>
Date: Mon, 14 Apr 2014 11:23:29 -0700
Message-Id: <8211784F-E4F1-4500-84FA-2FF3FC43C132@hortonworks.com>
References: <29230441-F926-4A2C-BE98-A5D29C813897@hortonworks.com>
To: general@incubator.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

The votes passes with:

8 binding +1, Sebastian Schelter, Henry Saputra, Suresh Srinivas, Till West=
mann, Andrew Purtell, Owen O=92Malley, Roman Shaposhnik, Jakob Homan

1 non-binding +1, Hitesh Shah

no +/-0 or -1s.

Congratulations to the new Stratosphere project.  I=92ll get started on the=
 project set up tasks.

Alan.

On Apr 10, 2014, at 10:42 AM, Alan Gates <gates@hortonworks.com> wrote:

> Based on the results of the discussion thread (http://mail-archives.apach=
e.org/mod_mbox/incubator-general/201403.mbox/%3CCE562EE9-968C-420E-A719-8C0=
8CDAC99F8%40hortonworks.com%3E in particular notice the discussion on name =
change in the disucssion ), I would like to call a vote on accepting Strato=
sphere into the incubator.
>=20
> [ ] +1 Accept Stratosphere into the Incubator
> [ ] +0 Indifferent to the acceptance of Stratosphere
> [ ] -1 Do not accept Stratosphere because =85
>=20
> The vote will be open until Monday April 14 18:00 UTC.
>=20
> https://wiki.apache.org/incubator/StratosphereProposal
>=20
> =3D Stratosphere =3D
> =3D=3D Abstract =3D=3D
> Stratosphere is an open source system for parallel data analysis. Stratos=
phere deeply integrates MapReduce and database technologies to provide expr=
essive and optimizable programming interfaces and at the same time efficien=
t and scalable execution.
>=20
> =3D=3D Proposal =3D=3D
> Stratosphere is an open source system for expressive, declarative, fast, =
and efficient data analysis. Stratosphere combines the scalability and prog=
ramming flexibility of distributed MapReduce-like platforms with the effici=
ency, out-of-core execution, and query optimization capabilities found in p=
arallel databases.
>=20
> =3D=3D Background =3D=3D
> There is currently a need for general-purpose cluster computing platforms=
 that are compatible with the Hadoop ecosystem, are more efficient, easier =
to use, and can support more applications than Hadoop MapReduce, but are no=
t restricted to a specific data model and language (such as the relational =
model and a variant of SQL). Stratosphere fulfils these needs.
>=20
> Stratosphere exposes expressive APIs in Java and Scala (conceptually simi=
lar to Spark, Cascading, Scalding) that allow arbitrary user-defined functi=
ons in the same language and data model that the program is written in. Str=
atosphere programs pass through a cost-based optimizer that finds the best =
execution path for these programs depending on the data and cluster charact=
eristics. The design and implementation of Stratosphere is based on researc=
h that generalizes query optimizers in relational databases. Stratosphere h=
as a distributed runtime that is architected upon the principles of paralle=
l databases, providing true pipelining (a basis for stream processing) and =
efficient out-of-core algorithms for grouping, sorting, joining, and aggreg=
ating data. Stratosphere provides first-class support for iterative algorit=
hms via a built-in iterate operator, covering Machine Learning and graph an=
alysis use cases. It achieves performance similar to Apache Giraph without =
being a specialized graph processing system.
>=20
> Stratosphere has undergone three major releases (v0.1, v0.2, v0.4) and so=
me minor ones.
>=20
> =3D=3D Rationale =3D=3D
> Stratosphere started out in 2008 as a research project by the Technical U=
niversity of Berlin, the Humboldt University of Berlin, and the Hasso Platt=
ner Institute, and has received subsequent funding from the German Research=
 Council, the European Institute of Innovation and Technology, the European=
 Commision, and industry.
>=20
> The traction of Stratosphere has by far exceeded our initial expectations=
, and we are therefore seeking an organizational long-term home for Stratos=
phere beyond the University walls that will house and further encourage con=
tributors from companies and other organizations that are interested in Str=
atosphere. We believe that the Apache Software Foundation is the ideal home=
 for Stratosphere. Stratosphere integrates with several existing Apache pro=
jects, such as HDFS, YARN, HBase, and Avro. The team is familiar with the A=
pache processes and fully subscribes to the Apache mission. One of the prop=
osing members is a long-time Apache contributor and PMC member.
>=20
> =3D=3D Initial Goals =3D=3D
> * Move the existing codebase to Apache
> * Integrate with the Apache development process
> * Ensure all dependencies are compliant with Apache License version 2.0
> * Incremental development and releases per Apache guidelines
>=20
> =3D=3D Current Status =3D=3D
> =3D=3D=3D Meritocracy =3D=3D=3D
> Stratosphere operated on meritocratic principles from the get go. The ini=
tial project proposal submitted to the German Research Council in 2008 stat=
ed that all code developed in the project will be released as open source u=
nder the Apache 2 license. Currently, all the discussions pertaining to Str=
atosphere development are public on [[https://github.com/stratosphere/strat=
osphere|GitHub]]  and our [[https://groups.google.com/forum/#!forum/stratos=
phere-dev|mailing list]]. The current incubation proposal includes the majo=
r code contributors to Stratosphere. Several additional people have worked =
on the Stratosphere codebase for research prototypes and industry use cases=
 and would be interested in becoming committers. We are starting with a sma=
ll committer group and we plan to add additional committers following an op=
en merit-based decision process during the incubation phase.
>=20
> =3D=3D=3D Community =3D=3D=3D
> Currently, the core of Stratosphere is developed at TU Berlin, mainly by =
the committers listed in this proposal. Additional people from several Univ=
ersities and companies in Europe are working with Stratosphere and are inte=
rested in becoming committers to the project.
>=20
> During the years, Stratosphere has been adopted as a platform for researc=
h and teaching in several Universities (TU Berlin, HU Berlin, HPI, RWTH, In=
ria, KTH, U. Trento, UCSD, and others), and it is currently witnessing its =
first industrial installations. We are seeing a rapidly growing interest in=
 Stratosphere by both startups and large companies, as well as a growing co=
mmunity (our first [[http://stratosphere.eu/events/2013/summit.html|Stratos=
phere Summit]] in November 2013 attracted over 80 participants). Stratosphe=
re was recently accepted as a mentoring organization in Google Summer of Co=
de 2014.
>=20
> We believe that acceptance in the Apache Software Foundation will consoli=
date the current community under one organizational umbrella, and most impo=
rtantly accelerate the growth of the community.
>=20
> =3D=3D=3D Core developers =3D=3D=3D
> The core developers of the system are Stephan Ewen, Fabian Hueske, Daniel=
 Warneke, Robert Metzger, Ufuk Celebi, and Aljoscha Krettek, who are all co=
mmitters in the current proposal.
>=20
> =3D=3D=3D Alignment =3D=3D=3D
> Stratosphere is compatible with, and related to several Apache projects. =
Stratosphere re-uses parts of Apache Hadoop, in particular HDFS and YARN, a=
s well as Apache HBase and Apache Avro. Stratosphere is a very good compila=
tion target for query languages such as Apache Hive and Apache Pig.
>=20
> =3D=3D Known Risks =3D=3D
> =3D=3D=3D Orphaned Products =3D=3D=3D
> There is strong interest in Stratosphere by several companies and organiz=
ations, and there is currently a long-term commitment to fund salaried deve=
lopers for Stratosphere by public and private organizations in Europe.
>=20
> =3D=3D=3D Inexperience with Open Source =3D=3D=3D
> Sebastian Schelter is a committer and PMC member of Apache Mahout and Apa=
che Giraph, member of the Apache Software Foundation, member of the Incubat=
or PMC and project mentor for Apache Drill. Sebastian, along with our mento=
rs, will guide the rest of the committers that have experience with releasi=
ng software as open source but little experience in participating in an ope=
n source project besides Stratosphere itself.
>=20
> In mid-2013 Stratosphere transitioned from an =93open source project with=
 publicly accessible source code=94 to an open source project that puts the=
 community first. We moved from a University-hosted git repository to GitHu=
b, where we discuss all issues publicly. This also includes release plannin=
g (via GitHub=92s milestone feature) and code reviews. We also moved our bu=
ild system to the publicly available Travis-CI. The mailing lists are hoste=
d with Google Groups, we use the public Maven repository infrastructure of =
Sonatype. The source code of the www.stratosphere.eu website is publicly av=
ailable and is meant to be changed by external contributors (for example fo=
r documentation purposes).
>=20
> =3D=3D=3D Homogeneous Developers =3D=3D=3D
> Most committers in this proposal belong to the same institution (TU Berli=
n). The engagement of these committers goes well beyond the necessary devel=
opment to support research, and all committers work on Stratosphere in thei=
r free time. Several people from other institutions are working on and are =
familiar with the Stratosphere codebase. We will work to attract them as fu=
ture committers during the incubation phase, following a merit-based approa=
ch.
>=20
> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
> Currently, Stratosphere receives support from salaried developers, in par=
ticular from graduate students at TU Berlin that are funded by the German R=
esearch Council, the European Institute of Technology, and the European Com=
mission. These students work in their free time on Stratosphere in addition=
 to their employment.
>=20
> We expect that Stratosphere development will occur on both salaried and v=
olunteer time. We will recruit additional committers, including non-salarie=
d developers, and we will work to ensure that the project will move forward=
 independently of salaried developers.
>=20
> =3D=3D=3D Relationship with Other Apache Products =3D=3D=3D
> Stratosphere interfaces with several existing Apache projects: Apache HBa=
se for storage, Apache Hadoop (HDFS for storage, YARN for resource manageme=
nt, and Stratosphere contains a generic wrapper for Hadoop MapReduce input =
formats), and Apache Avro (for serialization). Stratosphere uses Apache Mav=
en and Apache Commons libraries internally. Stratosphere can be a great com=
pilation target for Apache Pig and Apache Hive, although such functionality=
 is not yet implemented.
>=20
> Stratosphere is also related with several projects undergoing incubation =
in the Apache Incubation project, such as Tez, Drill, and Spark (graduated)=
. While all these projects target sufficiently different spaces and have di=
fferent architectures, it would be interesting to explore code reuse possib=
ilities. For example, we are currently basing our design for compiling SQL =
to Stratosphere on the Optiq library, also used by Apache Drill.
>=20
> =3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D
> We believe that the Apache brand will help us attract contributors to Str=
atosphere, by giving us a well-defined, transparent development process und=
er a known brand. At the same time, Stratosphere already has a healthy comm=
unity and current funding guarantees the further codebase development and g=
rowth of the project for the next 3-5 years. The reason for this proposal i=
s not to gain publicity, but to further strengthen the longevity of the pro=
ject as explained in the Rationale section.
>=20
> =3D=3D Documentation =3D=3D
> * [[https://stratosphere.eu|Project website]]
> * [[http://stratosphere.eu/docs/0.4/|Documentation]]
> * [[https://github.com/stratosphere/stratosphere|Codebase]]
> * [[https://groups.google.com/forum/#!forum/stratosphere-dev|Mailing list=
]]
>=20
> =3D=3D Initial Source =3D=3D
> Stratosphere is hosted on [[https://github.com/stratosphere/stratosphere|=
GitHub]] . This is the codebase that we will migrate to the Apache Foundati=
on. The code was previously hosted on a TU Berlin=92s own git infrastructur=
e. It has always been Apache 2.0 licensed.
>=20
> =3D=3D=3D Source and Intellectual Property Submission Plan =3D=3D=3D
> All initial and past committers will sign a CLA with the ASF while the in=
cubator proposal for Stratosphere is being discussed. All organizations tha=
t have employed Stratosphere contributors in the past will sign a SGA. Curr=
ent contributors will sign a CCLA. All major contributors are still active =
in the project.
>=20
> =3D=3D=3D External Dependencies =3D=3D=3D
> All critical dependencies are, to the extend of our knowledge, from other=
 Apache projects. These include Apache Hadoop (for YARN and HDFS) and some =
libraries (log4j, commons codec, junit and more). Our web frontend uses som=
e MIT-licensed JavaScript libraries.
>=20
> =3D=3D Required Resources =3D=3D
> =3D=3D=3D Mailing list =3D=3D=3D
> We will migrate our mailing lists to the following:
>=20
> * users@stratosphere.incubator.apache.org
> * dev@stratosphere.incubator.apache.org
> * private@stratosphere.incubator.apache.org
> * commits@stratosphere.incubator.apache.org
>=20
> =3D=3D=3D Source control =3D=3D=3D
> We would like to use Git for source control and enable GitHib mirroring f=
unctionality, where code reviews on GitHub are automatically forwarded to t=
he developer mailing list. (See also: https://blogs.apache.org/infra/entry/=
improved_integration_between_apache_and)
>=20
> =3D=3D=3D Issue tracking =3D=3D=3D
> We are currently using GitHub for issue tracking. We request an Apache-ho=
sted JIRA, and we will import existing issues there.
>=20
> =3D=3D Initial committers =3D=3D
> * Stephan Ewen - stephan.ewen@tu-berlin.de
> * Fabian Hueske - fabian.hueske@tu-berlin.de
> * Daniel Warneke - warneke@posteo.de
> * Robert Metzger - metrobert@gmail.com
> * Ufuk Celebi - u.celebi@fu-berlin.de
> * Aljoscha Krettek - aljoscha.krettek@gmail.com
> * Kostas Tzoumas - kostas.tzoumas@tu-berlin.de
> * Sebastian Schelter  - ssc@apache.org
>=20
> =3D=3D=3D Affiliations =3D=3D=3D
> * Stephan Ewen (TU Berlin)
> * Fabian Hueske (TU Berlin)
> * Daniel Warneke (Amadeus IT Group)
> * Robert Metzger (TU Berlin)
> * Ufuk Celebi (FU Berlin)
> * Aljoscha Krettek (TU Berlin)
> * Kostas Tzoumas (TU Berlin)
> * Sebastian Schelter (TU Berlin)
>=20
> =3D=3D Sponsors =3D=3D
> =3D=3D=3D Champion =3D=3D=3D
> Alan Gates ( gates@apache.org )
>=20
> =3D=3D=3D Nominated Mentors =3D=3D=3D
> * Sean Owen ( srowen@apache.org ) (Note: Sean is an Apache member but not=
 currently on the IPC, he will need to request IPMC membership)
> * Ted Dunning ( tdunning@apache.org )
> * Owen O'Malley ( omalley@apache.org )
> * Henry Saputra ( hsaputra@apache.org )
> * Ashutosh Chauhan (hashutosh@apache.org)
>=20
> =3D=3D=3D Sponsoring Entity =3D=3D=3D
> The Apache Incubator
>=20


--=20
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to=
=20
which it is addressed and may contain information that is confidential,=20
privileged and exempt from disclosure under applicable law. If the reader=
=20
of this message is not the intended recipient, you are hereby notified that=
=20
any printing, copying, dissemination, distribution, disclosure or=20
forwarding of this communication is strictly prohibited. If you have=20
received this communication in error, please contact the sender immediately=
=20
and delete it from your system. Thank You.

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org