Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Subject: Re: [VOTE] Accept Fluo into the Apache Incubator
From: Julian Hyde <jhyde@apache.org>
In-Reply-To: <CAF1jEfDkZr1bvN_qrx9qnAR2gX8H3u=ybufenPZaVjcCnjDJCg@mail.gmail.com>
Date: Fri, 13 May 2016 11:52:57 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <B0804594-364C-41E6-AE25-95DA550A9B84@apache.org>
References: <CAF1jEfDkZr1bvN_qrx9qnAR2gX8H3u=ybufenPZaVjcCnjDJCg@mail.gmail.com>
To: general@incubator.apache.org
archived-at: Fri, 13 May 2016 18:53:11 -0000

+1 (binding)

Julian


> On May 13, 2016, at 11:22 AM, Billie Rinaldi <billie@apache.org> =
wrote:
>=20
> Since discussion has died down, I would like to call a VOTE on =
accepting
> Fluo into the Apache Incubator.
>=20
> Proposal: http://wiki.apache.org/incubator/FluoProposal
>=20
> [ ] +1 Accept Fluo into the Apache Incubator
> [ ] +0 Abstain.
> [ ] -1 Do not accept Fluo into the Apache Incubator because=E2=80=A6
>=20
> This vote will be open for at least 72 hours.
>=20
> Here is my +1 (binding).
>=20
> Billie
> ------
>=20
> Fluo Proposal
>=20
> Abstract
>=20
> Fluo is a distributed system for incrementally processing large data =
sets
> stored in Accumulo.
>=20
> Proposal
>=20
> Fluo is a distributed transaction and notification system that enables =
the
> incremental processing of large data sets. Its transaction system =
allows
> for concurrent, cross-node updates to data stored in Accumulo. Its
> notification system enables developers to write code to be executed =
when
> observed data changes. Fluo provides a core API to perform =
transactional
> updates using minimalistic get/set methods. Fluo also provides a =
higher
> order recipes API that builds on the core API to support more complex
> methods for transactional updates.
>=20
> Background
>=20
> Several frameworks exist for batch (i.e Spark, MapReduce) and stream =
(i.e
> Storm, Spark Streaming) processing of data. While batch and stream
> processing have strong use cases, they are not suited for joining =
incoming
> data in real-time to a large existing data set. To fill this need, =
Google
> developed an incremental processing system called Percolator and =
described
> it in the paper, Large-scale Incremental Processing Using Distributed
> Transactions and Notifications1.
>=20
> Rationale
>=20
> Fluo fills the need for cross-row (and cross-node) transactions in =
Accumulo
> by providing it with an open source implementation of Percolator. Fluo =
also
> satisfies a gap in Accumulo=E2=80=99s ability to incrementally process =
data. Fluo
> also provides a novel recipes API which offers higher level =
abstractions
> for transactional updates.
>=20
> Current Status
>=20
> Fluo currently exists as an open source project on GitHub and has been =
in
> active development since 2013. The project has made an alpha release =
and
> two beta releases. The major features of Fluo outlined in this =
proposal
> have been implemented. Several example Fluo applications have been =
created
> and run successfully on clusters (up to 24 nodes).
>=20
> Meritocracy
>=20
> The Fluo project operates as a meritocracy and will continue to do so
> because we feel that a project comprised of a diverse set of =
committers
> will thrive. Therefore, we welcome new contributors and encourage them =
on
> their path to committership.
>=20
> Community
>=20
> Fluo is currently being used by a subset of the Accumulo community. =
The
> initial developers have been responsive to external contributions =
through
> pull requests and issues on GitHub. As Fluo releases a stable 1.0 =
version
> that is production-ready, we expect this community to grow. To =
encourage
> growth, we have created a project website with documentation, given =
talks
> at Meetups and the Accumulo Summit, and engaged with new users on =
GitHub
> and the Fluo mailing list.
>=20
> Core Developers
>=20
> The project was started by Keith Turner (an Apache Member and =
committer/PMC
> on Gora and Accumulo) in 2013, and the development has primarily =
consisted
> of his and Mike Walch=E2=80=99s continued efforts. Additional =
developers have
> contributed over time, which has led to new committers.
>=20
> Alignment
>=20
> Fluo is closely linked to the Accumulo community, and fits well within =
the
> larger Hadoop ecosystem at Apache. Fluo utilizes several Apache =
projects,
> such as Accumulo, YARN, Twill, and ZooKeeper. Enabling closer =
collaboration
> between these communities through its coexistence within the ASF would =
help
> further drive the success of them all.
>=20
> In addition to our technical ties to other ASF projects, our =
development
> philosophy aligns with Apache philosophies. Based on our experience =
with
> existing Apache projects, we are interested in establishing formal
> governance with a PMC and community bylaws, which we feel would best =
be
> done within Apache.
>=20
> Known Risks
>=20
> Orphaned Products
>=20
> Fluo could be orphaned if the project fails to gain adoption and the =
core
> developers abandon their interest (this is not anticipated). This risk =
can
> be mitigated by attracting more committers and developing further
> documentation to ease adoption.
>=20
> Inexperience with Open Source
>=20
> Fluo has been an open source project on GitHub from the start of its
> development. Several Fluo developers are committers on other ASF =
projects
> as well as open source projects outside ASF, and understand open =
source
> development.
>=20
> Homogeneous Developers
>=20
> The initial committers work for different employers. We hope add more
> developers from other employers and industries.
>=20
> Reliance on Salaried Developers
>=20
> While most of the initial committers are paid to work on Fluo, there =
have
> been many contributions from developers working independently.
>=20
> Relationships with Other Apache Products
>=20
> Fluo uses Accumulo, Hadoop (HDFS & YARN), Twill, ZooKeeper, Curator,
> Thrift, and various Commons libraries. During development, =
contributions
> have been made to some of these Apache projects to better support Fluo =
use
> cases.
>=20
> Apache Brand
>=20
> While we recognize the impact of the Apache brand, we feel that Fluo =
would
> fit well in Apache because of its relationship to other Apache =
projects and
> because we share the ASF values of meritocracy and community over =
code.
>=20
> Documentation
>=20
> Information about Fluo can be found on the project website at
> http://fluo.io/. This includes:
>=20
>    General documentation - http://fluo.io/docs/
>=20
>    API documentation - http://fluo.io/apidocs/
>=20
>    Release notes - http://fluo.io/release-notes/
>=20
>    Blog posts - http://fluo.io/blog/
>=20
> Initial Source
>=20
> The initial source code is publicly available as an open source =
project on
> GitHub at https://github.com/fluo-io/fluo
>=20
> Supplemental repositories also exist on GitHub at =
https://github.com/fluo-io
> and some of those will become part of the initial code base (perhaps =
in
> separate repositories).
>=20
> Source and Intellectual Property Submission Plan
>=20
> All of the Fluo=E2=80=99s source code is available under the Apache =
License,
> Version 2.
>=20
> The Fluo logo was designed and contributed to the Fluo project, for =
use by
> the project, and the contributors would like it to remain the logo of =
the
> project within the ASF, granting any necessary rights to the ASF, =
while
> continuing to use the logo on Fluo-related historical sites and =
project
> pages (such as Fluo=E2=80=99s current GitHub site).
>=20
> External Dependencies
>=20
> Fluo has made it a point from its beginning to use dependencies which =
are
> compatible with the expectations of an ASF project. The following are =
its
> current dependencies, grouped by license.
>=20
> Apache License, Version 2.0
>=20
>    accumulo
>    commons-{collections,configuration,io}
>    curator
>    dropwizard metrics
>    easymock
>    guava
>    hadoop
>    jcommander
>    maven
>    thrift
>    twill
>    zookeeper
>=20
> BSD License (2-Clause)
>=20
>    HdrHistogram
>=20
> Eclipse Public License - v 1.0
>=20
>    junit (not bundled)
>    logback (binary bundling only)
>=20
> MIT License (Expat)
>=20
>    slf4j
>=20
> Cryptography
>=20
> none
>=20
> Required Resources
>=20
> Mailing Lists
>=20
>    private at fluo.incubator.apache.org
>    dev at fluo.incubator.apache.org
>    notifications at fluo.incubator.apache.org
>=20
> Git Repository
>=20
>    https://git-wip-us.apache.org/repos/asf/incubator-fluo.git
>=20
>        (The developers will use a git-based site for project =
documentation
> in the asf-site branch of the repo.)
>=20
>    https://git-wip-us.apache.org/repos/asf/incubator-fluo-recipes.git
>=20
> Issue Tracking
>=20
>    https://issues.apache.org/jira/browse/FLUO
>=20
>        (Currently, the developers rely on GitHub issues. If possible,
> GitHub integration for issue tracking would be preferred. If this is
> possible, the Fluo developers could work with INFRA to transfer the
> existing GitHub repositories to the Apache GitHub organization to =
bring the
> existing GitHub issues.)
>=20
> Continuous Integration
>=20
>    Travis CI on the GitHub mirror is fine (flag set to build only if
> .travis.yml file is present)
>=20
> Initial Committers
>=20
>    Keith Turner (kturner at apache dot org)
>    Mike Walch (mike.walch at ptech-llc dot com)
>    Corey Nolet (cjnolet at apache dot org)
>    Christopher Tubbs (ctubbsii at apache dot org)
>    Josh Elser (elserj at apache dot org)
>=20
> Affiliations
>=20
>    Keith Turner (Peterson Technologies, ASF Member, Accumulo PMC, Gora =
PMC)
>    Mike Walch (Peterson Technologies)
>    Corey Nolet (Tetra Concepts LLC, Accumulo PMC)
>    Christopher Tubbs (U.S. Government, ASF Member, Accumulo PMC)
>    Josh Elser (Hortonworks, ASF Member, Accumulo PMC, Calcite PMC, =
IPMC)
>=20
> Sponsors
>=20
> Champion
>=20
>    Billie Rinaldi (billie at apache dot org) has volunteered to be our
> Champion
>=20
> Nominated Mentors
>=20
>    Drew Farris (drew at apache dot org)
>    Josh Elser (elserj at apache dot org)
>    Billie Rinaldi (billie at apache dot org)
>=20
> Sponsoring Entity
>=20
>    The Fluo team requests sponsorship from the Incubator PMC
>=20
>    USENIX (2010), http://research.google.com/pubs/pub36726.html (1)


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org