Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of esammer@cloudera.com designates
 209.85.210.47 as permitted sender)
MIME-Version: 1.0
Sender: esammer@cloudera.com
In-Reply-To: <BANLkTik2Gfd1pZwXgPUKQUW62f2m5R3O-A@mail.gmail.com>
References: <BANLkTik2Gfd1pZwXgPUKQUW62f2m5R3O-A@mail.gmail.com>
Date: Fri, 17 Jun 2011 12:05:54 -0700
Message-ID: <BANLkTinxmirwBz7D76cGvVXmfM7LeARsTQ@mail.gmail.com>
Subject: Re: [VOTE] Accept Bigtop for incubation
From: Eric Sammer <esammer@apache.org>
To: general@incubator.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

+1 (non-binding).

On Fri, Jun 17, 2011 at 10:15 AM, Tom White <tomwhite@apache.org> wrote:
> As there are no active discussions on the proposal thread, I would
> like to initiate a vote to accept Bigtop as an Apache Incubator
> project.
>
> The proposal is available at
>
> http://wiki.apache.org/incubator/BigtopProposal?action=3Drecall&rev=3D13
>
> I've also put a copy of the proposal at the end of this email.
>
> The discussion thread is available at
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201106.mbox/%3=
CBANLkTimriyVS5G5MAKLQviNAUZ9H6S5hWw@mail.gmail.com%3E
>
> Please cast your votes:
>
> [ =A0] +1 Accept Bigtop for incubation
> [ =A0] +0 Indifferent to Bigtop incubation
> [ =A0] -1 Reject Bigtop for incubation
>
> This vote will close 72 hours from now.
>
> Thanks,
> Tom
>
> =3D Bigtop - Apache Hadoop Ecosystem Packaging and Test =3D
>
> =3D=3D Abstract =3D=3D
>
> Bigtop - a project for the development of packaging and tests of the
> Hadoop ecosystem.
>
> =3D=3D Proposal =3D=3D
>
> The primary goal of Bigtop is to build a community around the
> packaging and interoperability testing of Hadoop-related projects.
> This includes testing at various levels (packaging, platform, runtime,
> upgrade, etc...) developed by a community with a focus on the system
> as a whole, rather than individual projects.
>
> Build, packaging and integration test code that depends upon official
> releases of the Apache Hadoop-related projects (HDFS, MapReduce,
> HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by
> this project. As bugs and other issues are found we expect these to be
> fixed upstream.
>
> =3D=3D Background =3D=3D
>
> The initial packaging and test code for Bigtop was developed by
> Cloudera to package projects from the Apache Hadoop ecosystem and
> provide a consistent, inter-operable framework.
>
> =3D=3D Rationale =3D=3D
>
> Hadoop defines itself as:
>
> {{{
> The Apache Hadoop project develops open-source software for reliable,
> scalable, distributed computing. Hadoop includes these subprojects:
>
> * Hadoop Common: The common utilities that support the other Hadoop subpr=
ojects.
> * HDFS: A distributed file system that provides high throughput access
> to application data.
> * MapReduce: A software framework for distributed processing of large
> data sets on compute clusters.
> }}}
>
> There are also several other Hadoop-related projects at Apache. =A0Some
> TLP examples include HBase, Hive, Mahout, ZooKeeper, and Pig. =A0There
> are also several new projects in the Incubator such as HCatalog, Hama
> and Sqoop.
>
> From a packaging and deployment perspective, the current
> loosely-coupled nature of the project has limitations:
> =A01. Insufficient building against trunk versions of dependent projects
> (in the style of Apache Gump).
> =A01. Insufficient testing against the trunk versions of dependent projec=
ts.
> =A01. No consistent packaging for the Linux servers which provide the
> main Hadoop datacenter platform.
> =A01. No functional testing against multi-machine clusters as part of
> the regular automated build process. This is due to a lack of a
> physical or virtual Hadoop cluster for testing, and not enough test
> suites designed to run against a live cluster with known datasets.
>
> The intent of this project is to build a community where the projects
> are brought together, packaged, and tested for interoperability.
>
> Projects such as Apache Whirr (incubating), which deploy and use a
> collection of Hadoop-related projects, would benefit from the
> interoperability testing done by Bigtop, rather than picking and
> testing project combinations themselves.
>
> =3D=3D Initial Goals =3D=3D
>
> Much of the code for Bigtop has been released by Cloudera under the
> Apache 2.0 license for over two years.
>
> Some current goals include:
> =A0* create a set of packages for the Hadoop ecosystem, over a wide
> range of platforms
> =A0* interoperability test these projects
> =A0* document project sets that are known to work well together
>
> Bigtop=92s release artifact would consist of a single tarball of
> packaging and test code that, when built, would produce source and
> binary Linux packages for the upstream projects.
>
> =3D Current Status =3D
>
> =3D=3D Meritocracy =3D=3D
>
> Bigtop was originally developed and released as an open source
> packaging infrastructure, CDH, by Cloudera.
>
> =3D=3D Community =3D=3D
>
> The community is primarily the original developers at Cloudera,
> however a number of contributions to the packaging specifications have
> been accepted from outside contributors. Growing a diverse community
> is the main reason to bring Bigtop to the Apache Incubator.
>
> =3D=3D Core Developers =3D=3D
>
> The core developers for Bigtop project are:
> =A0* Andrew Bayer has extensive expertise with build tools, specifically
> Jenkins continuous integration and Maven.
> =A0* Peter Linnell has contributed to the RPM packaging.
> =A0* Bruno Mah=E9 has overseen much of the development of the RPM and
> Debian packaging system.
> =A0* Roman Shaposhnik and Konstantin Boudnik designed and implemented
> the system testing framework.
>
> Many of the committers to the Bigtop project have contributed towards
> Hadoop or related Apache projects (Alejandro Abdelnur, Konstantin
> Boudnik, Eli Collins, Alan Gates, Patrick Hunt, Steve Loughran, Owen
> O'Malley, John Sichi, Michael Stack, Tom White) and are familiar with
> Apache principals and philosophy for community driven software
> development.
>
> =3D=3D Alignment =3D=3D
>
> We expect projects in Bigtop to be drawn from Hadoop and related
> projects at Apache. Bigtop will complement these projects (Hadoop,
> Pig, Hive, HBase, etc...) by providing an environment for contributors
> interested in building more complex data processing pipelines to work
> together integrating more than a single project into a well-tested
> whole.
>
> =3D Known Risks =3D
>
> =3D=3D Orphaned Products =3D=3D
>
> The contributors are leading vendors of Hadoop-based technologies and
> have a long standing in the Hadoop community. There is minimal risk of
> this work becoming non-strategic and the contributors are confident
> that a larger community will form within the project in a relatively
> short space of time.
>
> =3D=3D Inexperience with Open Source =3D=3D
>
> All code developed for Bigtop has been open sourced under the Apache
> 2.0 license. Most committers of Bigtop project are intimately familiar
> with the Apache model for open-source development and are experienced
> with working with new contributors.
>
> =3D=3D Homogeneous Developers =3D=3D
>
> The initial set of committers is from a small set of organizations and
> numerous existing Apache projects. We expect that once approved for
> incubation, the project will attract new contributors from more
> organizations and will thus grow organically.
>
> =3D=3D Reliance on Salaried Developers =3D=3D
>
> It is expected that Bigtop will be developed on salaried and volunteer
> time, although all of the initial developers will work on it mainly on
> salaried time.
>
> =3D=3D Relationships with Other Apache Products =3D=3D
>
> Bigtop depends upon other Apache Projects including Apache Hadoop,
> Apache HBase, Apache Hive, Apache Pig, Apache Zookeeper, Apache
> Thrift, Apache Avro, Apache Whirr. The build system uses Apache Ant
> and Apache Maven.
>
> =3D=3D An Excessive Fascination with the Apache Brand =3D=3D
>
> We would like Bigtop to become an Apache project to further foster a
> healthy community of contributors and consumers around
> interoperability, testing and packaging of Hadoop projects. Since
> Bigtop directly interacts with many Apache Hadoop-related projects and
> solves important problems of many Hadoop users, residing in the the
> Apache Software Foundation will increase interaction with the larger
> community.
>
> =3D Documentation =3D
>
> =A0* Bigtop will develop its own documentation detailing how to build,
> test, install, configure and debug.
>
> =3D Initial Source =3D
>
> =A0* https://github.com/cloudera/bigtop
>
> =3D=3D Source and Intellectual Property Submission Plan =3D=3D
>
> =A0* The initial source is already licensed under the Apache License, Ver=
sion 2.0.
>
> https://github.com/cloudera/bigtop
>
> =3D=3D External Dependencies =3D=3D
>
> The required external dependencies are all Apache License or
> compatible licenses.
>
> =3D=3D Cryptography =3D=3D
>
> Bigtop doesn't use cryptography itself, however Hadoop projects use
> standard APIs and tools for SSH and SSL communication where necessary.
>
> =3D Required =A0Resources =3D
>
> =3D=3D Mailing lists =3D=3D
>
> =A0* bigtop-private (with moderated subscriptions)
> =A0* bigtop-dev
> =A0* bigtop-commits
> =A0* bigtop-user
>
> =3D=3D Subversion Directory =3D=3D
>
> https://svn.apache.org/repos/asf/incubator/bigtop
>
> =3D=3D Issue Tracking =3D=3D
>
> JIRA BIGTOP (Bigtop)
>
> =3D=3D Other Resources =3D=3D
>
> The existing code already has unit and integration tests so we would
> like a Jenkins instance to run them whenever a new patch is submitted.
> This can be added after project creation.
>
> To test RPM & deb install/uninstall and upgrade, it is useful to have
> a set of Virtual Machine images in known states, and servers that can
> bring them up. It should be possible to use Apache Whirr to
> choreograph the VM setup/teardown, so these tests could be performed
> against VMs on developer desktops or large scale VM-hosting platforms.
> For the latter, VM hosting time would be appreciated.
>
> =3D Initial Committers =3D
>
> =A0* Alejandro Abdelnur (tucu at cloudera dot com)
> =A0* Andre Arcilla (arcilla at yahoo-inc dot com)
> =A0* Andrew Bayer (abayer at cloudera dot com)
> =A0* Konstantin Boudnik (cos at apache dot org)
> =A0* Eli Collins (eli at apache dot org)
> =A0* Travis Crawford (travis at twitter dot com)
> =A0* Bruno Mah=E9 (bruno at cloudera dot com)
> =A0* Alan Gates (gates at apache dot org)
> =A0* Patrick Hunt (phunt at apache dot org)
> =A0* Peter Linnell (plinnell at cloudera dot com)
> =A0* Steve Loughran (stevel at apache dot org)
> =A0* Owen O'Malley (omalley at apache dot org)
> =A0* James Page (James.page at canonical dot com)
> =A0* Roman Shaposhnik (rvs at cloudera dot com)
> =A0* John Sichi (jvs at apache dot org)
> =A0* Michael Stack (stack at apache dot org)
> =A0* Tom White (tomwhite at apache dot org)
> =A0* Andrei Savu (asavu at apache dot org)
> =A0* Edward J. Yoon (edwardyoon at apache dot org)
>
> =3D Affiliations =3D
>
> =A0* Alejandro Abdelnur, Cloudera
> =A0* Andre Arcilla, Yahoo! Inc.
> =A0* Andrew Bayer, Cloudera
> =A0* Konstantin Boudnik, free lancer
> =A0* Eli Collins, Cloudera
> =A0* Travis Crawford, Twitter
> =A0* Bruno Mah=E9, Cloudera
> =A0* Alan Gates, Yahoo!
> =A0* Patrick Hunt, Cloudera
> =A0* Peter Linnell, Cloudera
> =A0* Steve Loughran, HP Laboratories
> =A0* Owen O'Malley, Yahoo!
> =A0* James Page, Canonical
> =A0* Roman Shaposhnik, Cloudera
> =A0* John Sichi, Facebook
> =A0* Michael Stack, StumbleUpon
> =A0* Tom White, Cloudera
> =A0* Andrei Savu, Adobe
> =A0* Edward J. Yoon, Korea Telecom
>
> =3D Sponsors =3D
>
> =3D=3D Champion =3D=3D
>
> =A0* Patrick Hunt
>
> =3D=3D Nominated Mentors =3D=3D
>
> =A0* Patrick Hunt
> =A0* Tom White
> =A0* Owen O'Malley
> =A0* Alan Gates
> =A0* Steve Loughran
>
> =3D=3D Sponsoring Entity =3D=3D
>
> =A0* Apache Incubator PMC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org