Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
From: "Debo Dutta (dedutta)" <dedutta@cisco.com>
To: "general@incubator.apache.org" <general@incubator.apache.org>
Subject: Re: [VOTE] Accept Spot into the Apache Incubator
Thread-Topic: [VOTE] Accept Spot into the Apache Incubator
Thread-Index: AQHSE2r+TYjiVYqT0k27C0zLBnQB2KCFer0AgAAsIQY=
Date: Thu, 22 Sep 2016 15:34:48 +0000
Message-ID: <CE64976B-3A90-448B-8FF9-561534A8411A@cisco.com>
References: <CALEq1Z9FZpdU3P+8nOvA0UTMnrOi9Rb5-DrByiDibqyHNuocWw@mail.gmail.com>,<CABqR8u_BErg=kVctN64gB+E6JQF+jYhweL77sRe5sjJXsULOag@mail.gmail.com>
In-Reply-To: <CABqR8u_BErg=kVctN64gB+E6JQF+jYhweL77sRe5sjJXsULOag@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
archived-at: Thu, 22 Sep 2016 15:34:58 -0000

+1

Sent from my iPhone

> On Sep 22, 2016, at 12:57 AM, Tom White <tom.e.white@gmail.com> wrote:
>=20
> +1
>=20
> Tom
>=20
>> On Tue, Sep 20, 2016 at 7:15 PM, Doug Cutting <cutting@apache.org> wrote=
:
>> Following the discussion thread, I would like to call a vote on
>> accepting Spot into the Apache Incubator.
>>=20
>> [] +1 Accept Spot into the Apache Incubator
>> [] +0 Abstain.
>> [] -1 Do not accept Spot into the Apache Incubator because ...
>>=20
>> This vote will run for the usual 72 hours.
>>=20
>> The proposal is attached, but you can also access it on the wiki:
>>   https://wiki.apache.org/incubator/SpotProposal
>>=20
>> Thanks,
>>=20
>> Doug
>>=20
>> =3D SpotProposal =3D
>>=20
>> =3D=3D Abstract =3D=3D
>>=20
>> Spot is an open source platform for network telemetry (packet, flow,
>> and proxy at the moment) built on an open data model and Apache
>> Hadoop.
>>=20
>> =3D=3D Proposal =3D=3D
>>=20
>> Spot (formerly Open Network Insight, or ONI) is an open source
>> solution for network telemetry (packet, flow, and proxy at the moment)
>> built on an open data model and Apache Hadoop. It provides ingestion
>> and transformation of binary data, scalable machine learning, and
>> interactive visualization for identifying threats in network flows and
>> DNS packets.
>>=20
>> Spot has a pluggable architecture that can accommodate multiple open
>> data models. Although cybersecurity/network-intrusion analysis is the
>> initial use case for Spot, we are actively encouraging the
>> contribution of new models that will enable other adjacent
>> applications, such as fraud detection or IT-operational analytics such
>> as performance and health monitoring. Because these models are open,
>> users maintain control of their own data.
>>=20
>> More information on Spot can be found at the existing project website
>> at http://open-network-insight.org/.
>>=20
>> =3D=3D Background =3D=3D
>>=20
>> It almost goes without saying that cybersecurity is an acute and
>> paramount concern globally, for organizations of all types and
>> sizes. Fortunately, thanks to the availability of massively scalable
>> (in the PBs) data infrastructure, security professionals can now make
>> authentically data-driven decisions about how they protect their
>> assets. For example, records of network traffic, captured as network
>> flows, are often stored and analyzed for use in network management,
>> and this same information can provide valuable insights into network
>> vulnerabilities.
>>=20
>> Cybersecurity is just one example, however: There are other examples
>> of adjacent use cases, such as user fraud detection or IT-operations
>> analytics, that would benefit from the combination of Spot
>> functionality and PB-scale data sets for analysis.
>>=20
>> =3D=3D Rationale =3D=3D
>>=20
>> Although cybersecurity is its initial use case/data model, Spot is
>> intended to more generally tackle the dual challenges of facilitating
>> the development of big data-driven analytic solutions, while helping
>> vendors avoid having to create one/off infrastructure for each use
>> case. Spot will eliminate issues related to vendor data models that
>> create silos between solutions, and that make it difficult for users
>> to consume these innovations from multiple vendors. In summary, Spot
>> will accelerate the development of new massively scalable analytic
>> applications that give users more flexibility, and more choices.
>>=20
>> As an initial effort, we are now seeking to build an ecosystem of
>> developers, data scientists, and security professionals to make Spot
>> the open, community-driven, cybersecurity platform standard it needs
>> to become. By bringing Spot to Apache, we hope to galvanize these
>> groups to cooperate in this highly matrixed effort, and to build a
>> global, and diverse, Spot community.
>>=20
>> =3D=3D Initial Goals =3D=3D
>>=20
>> Move the existing codebase, website, documentation, and mailing lists
>> to Apache-hosted infrastructure Work with the infrastructure team to
>> implement and approve our build and testing workflows in the context
>> of the ASF Incremental development and releases per Apache guidelines
>>=20
>> =3D=3D Current Status =3D=3D
>>=20
>> =3D=3D=3D Releases =3D=3D=3D
>>=20
>> Spot has undergone one public release (1.0). This initial release was
>> not performed in the typical ASF fashion; we will adopt the ASF source
>> release process upon joining the incubator.
>>=20
>> =3D=3D=3D Source =3D=3D=3D
>>=20
>> Spot=92s source, including core platform and associated submodules, is
>> currently hosted in several GitHub repositories under the indicated
>> licenses:
>>=20
>> * Core (Apache License 2.0)
>> * Oni-ingest (Apache License 2.0)
>> * Oni-ml (Apache License 2.0
>> * Oni-oa (BSD & MIT)
>> * Oni-setup (Apache License 2.0)
>> * Oni-nfdump (BSD)
>> * Oni-lda-c (GNU General Public License version 2)
>>=20
>> The repositories will be transitioned to Apache=92s git hosting during
>> incubation.  Issues related to GPL code will be resolved during
>> incubation.
>>=20
>>=20
>> =3D=3D=3D Issue Tracking =3D=3D=3D
>>=20
>> Spot=92s bug and feature tracking is hosted on Github at:
>>=20
>> * https://github.com/Open-Network-Insight/open-network-insight/issues
>>=20
>> Issue tracking will be transitioned to Apache=92s JIRA instance during i=
ncubation.
>>=20
>> =3D=3D=3D Code review =3D=3D=3D
>>=20
>> Spot maintainers currently use =93LGTM=94 (Looks Good to Me) in comments
>> on the code review to indicate acceptance, with at least three LGTMs
>> required to approve the merge.
>>=20
>> =3D=3D=3D Community discussion =3D=3D=3D
>>=20
>> A Spot Slack channel is available at:
>>=20
>> * https://opennetworkinsights.slack.com/messages/general/ (Invites
>> request via http://open-network-insight.org:3000/)
>>=20
>> Community discussion options will be expanded considerably when
>> apache.org mailing lists are available.
>>=20
>> =3D=3D=3D Meritocracy =3D=3D=3D
>>=20
>> We intend to adhere to a meritocratic approach to electing new
>> committers and PMC members. We also believe that contributions can
>> come in forms other than just code. We will encourage contributions
>> and participation of all types, and ensure that contributors are
>> appropriately recognized and that PMC memberships are appropriately
>> earned.
>>=20
>> =3D=3D=3D Community =3D=3D=3D
>>=20
>> Though Spot is a relatively new project, it has already seen promising a=
doption:
>>=20
>> * Intel is the original development sponsor for Spot.
>> * Cloudera is strong advocate for open source cybersecurity solutions
>> and Apache Hadoop, and a supporter of Spot.
>> * Cloudwick=92s OAS cybersecurity solution is built on Spot.
>> * Accenture=92s Cyber Intelligence Platform solution is built on Spot.
>> * Centrify has announced its intention to contribute identity-based
>> security features to Spot=92s network-intrusion detection data model.
>> * Webroot has announced its intention to contribute endpoint-security
>> functionality.
>> * Cybraics has announced its intention to contribute network-security
>> functionality.
>> * Jask has announced its intention to contribute network-security
>> functionality.
>>=20
>> As described in the =93Rationale=94 section, we believe that building on
>> and expanding the Spot community will be a key aspect in its success.
>>=20
>> =3D=3D=3D Core Developers =3D=3D=3D
>>=20
>> Spot was initially developed as a project at Intel, and most of the
>> contributions to date have been from developers employed by that
>> company. By bringing Spot to Apache, we hope to diversify its
>> developer community more rapidly.
>>=20
>> =3D=3D=3D Alignment =3D=3D=3D
>>=20
>> Spot is built on Apache Hadoop, Apache Kafka, and Apache Spark, and as
>> more functionality is built out, integration with other Apache
>> projects is very likely.
>>=20
>> =3D=3D Known Risks =3D=3D
>>=20
>> =3D=3D=3D Orphaned products =3D=3D=3D
>>=20
>> The risk of Spot being abandoned is low. Intel has made substantial
>> investments already, Cloudera has publicly expressed the importance of
>> Spot as a =93killer app=94 for Apache Hadoop, and Cloudwick and Accentur=
e
>> both have offerings that are built on Spot/CDH.
>>=20
>> =3D=3D=3D Inexperience with Open Source =3D=3D=3D
>>=20
>> Most of Spot=92s initial committers have experience in open source
>> development, although not necessarily within the ASF. Those Spot
>> developers who have little open source experience or are not Apache
>> committers are eager to learn ASF practices as a means of improving
>> project governance and diversifying the developer community.
>>=20
>> =3D=3D=3D Homogenous Developers =3D=3D=3D
>>=20
>> As mentioned previously, Intel developers are mostly responsible for
>> what Spot code exists, to date. As a benefit of ASF governance, we
>> hope to scale-up contributions from new developers and community
>> members and eventually, develop them into committers by adhering to
>> Apache=92s meritocratic principles.
>>=20
>> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
>>=20
>> To date, all Spot code has been written by salaried developers
>> (chiefly employed by Intel).
>>=20
>> =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D
>>=20
>> Spot is currently related to the following other Apache projects:
>>=20
>> * Apache Hadoop
>> * Apache Spark
>> * Apache Kafka
>>=20
>> We look forward to continuing to integrate and collaborate with these
>> communities.
>>=20
>> =3D=3D=3D A Excessive Fascination with the Apache Brand =3D=3D=3D
>>=20
>> Although most (not all) of the initial committers are not currently
>> Apache committers, they are resolved to learning, with the help of the
>> more experienced committers/project mentors/champion, the Apache
>> Way. We believe that adhering to these principles will be of great
>> value with respect to meeting long-term project goals, including
>> facilitating widespread adoption.
>>=20
>> =3D=3D Documentation =3D=3D
>>=20
>> Spot functionality is divided into different repositories, with each
>> repository containing the relevant developer documentation:
>>=20
>> * oni-ingest
>> * oni-ml
>> * oni-oa
>> * oni-setup
>> * oni-nfdump
>> * oni-lda-c
>>=20
>> An Installation Guide is published in the project wiki:
>> * https://github.com/Open-Network-Insight/open-network-insight/wiki
>> The Spot (currently Open Network Insight) website is managed via a
>> Wordpress instance hosted by Bluehost:
>> * http://open-network-insight.org/
>> A Docker-based demo is available via Docker Hub:
>> * https://hub.docker.com/r/opennetworkinsight/oni-demo/
>>=20
>> =3D=3D Initial Source =3D=3D
>>=20
>> The Spot codebase is currently hosted on GitHub and will be
>> transitioned to the ASF repositories during incubation. Spot and its
>> submodules are currently licensed under several different licenses.
>>=20
>> No trademarks or domain names for Spot have been registered to date,
>> and it will be up to the ASF=92s discretion to do so. The project=92s
>> current website at open-network-insight.org will be redirected to
>> spot.incubator.apache.org during incubation.
>>=20
>> Some portions of the code are imported from other open source projects
>> under the Apache 2.0, BSD, or MIT licenses.
>>=20
>> =3D=3D External Dependencies =3D=3D
>>=20
>> The full set of dependencies and licenses are:
>> * Jupyter: BSD
>> * D3js: BSD
>> * Nfdump: BSD
>> * Wireshark: GNU General Public License version 2
>> * Apache Hadoop: Apache License 2.0
>> * Apache Spark: Apache License 2.0
>> * JQuery: MIT
>> * ReactJS: BSD
>> * Bootstrap: MIT
>>=20
>> Issues related to GPL dependencies will be resolved during incubation.
>>=20
>> =3D=3D Cryptography =3D=3D
>>=20
>> Spot does not currently include any cryptography-related code.
>>=20
>> =3D=3D Required Resources =3D=3D
>>=20
>> =3D=3D=3D Developer and user mailing lists =3D=3D=3D
>>=20
>> * private@spot.incubator.apache.org (PMC)
>> * commits@spot.incubator.apache.org (git push emails)
>> * issues@spot.incubator.apache.org (JIRA issue feed)
>> * dev@spot.incubator.apache.org (code reviews plus dev discussion)
>> * user@spot.incubator.apache.org (user questions)
>>=20
>> =3D=3D=3D Repository =3D=3D=3D
>>=20
>> * git://git.apache.org/spot
>>=20
>> =3D=3D=3D Issue Tracker =3D=3D=3D
>>=20
>> We would like to import our current JIRA project into the ASF JIRA,
>> such that our historical commit messages and code comments continue to
>> reference the appropriate bug numbers.
>>=20
>> =3D=3D Initial Committers =3D=3D
>>=20
>> * Grant Babb
>> * Ricardo Barona
>> * Cesar Berho
>> * Jarek Jarcec Cecho
>> * Michael Czerny
>> * Nick Gamb
>> * Sai Ganji
>> * Gabriela Lima Garza
>> * Victor Gonzalez
>> * Mark Grover
>> * Morris Hicks
>> * Ritu Kama
>> * Austin Leahy
>> * Ashrith Mekala
>> * Diego Ortiz
>> * Sudharshan Rao PakalaSai
>> * Srinivasa Reddy
>> * Alan Ross
>> * Everardo Lopez Sandoval
>> * Nathan Segerlind
>> * Vartika Singh
>> * Nathanael Smith
>> * Carlos Villavicencio
>>=20
>> =3D=3D Affiliations =3D=3D
>>=20
>> * Grant Babb: Jask
>> * Ricardo Barona : Intel
>> * Cesar Berho: Intel
>> * Jarek Jarcec Cecho: StreamSets
>> * Michael Czerny: Cybraics
>> * Nick Gamb: Centrify
>> * Sai Ganji: Cloudwick
>> * Gabriela Lima Garza: Intel
>> * Victor Gonzalez: Intel
>> * Mark Grover: Cloudera
>> * Morris Hicks: Cloudera
>> * Ritu Kama: Intel
>> * Austin Leahy: eBay
>> * Ashrith Mekala: Cloudwick
>> * Diego Ortiz: Intel
>> * Sudharshan Rao PakalaSai: Cloudwick
>> * Srinivasa Reddy: Cloudera
>> * Alan Ross: Intel
>> * Everardo Lopez Sandoval: Intel
>> * Nathan Segerlind: Intel
>> * Vartika Singh: Cloudera
>> * Nathanael Smith: Intel
>> * Carlos Villavicencio: Intel
>>=20
>> =3D=3D Sponsors =3D=3D
>>=20
>> =3D=3D=3D Champion =3D=3D=3D
>>=20
>> * Doug Cutting - Cloudera
>>=20
>> =3D=3D=3D Nominated Mentors =3D=3D=3D
>>=20
>> * Brock Noland - ASF Member, phData
>> * Jarek Jarcec Cecho - ASF Member, StreamSets
>> * Andrei Savu - Cloudera
>> * Uma Maheswara Rao G - Intel
>>=20
>> =3D=3D=3D Sponsoring Entity =3D=3D=3D
>>=20
>> The Apache Incubator.
>>=20
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>=20

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org