incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [PROPOSAL] NiFi for Incubation
Date Thu, 20 Nov 2014 04:11:57 GMT
This is *fan freakinĀ¹ tastic* Sounds like an awesome  project and
glad to hear a relationship to Tika! Awesome to see more government
projects coming into the ASF!

you already have a great set of mentors and I donĀ¹t really have more
time on my plate, but really happy and will try and monitor and help
on the lists.

Cheers!

Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++






-----Original Message-----
From: Joe Witt <joe.witt@gmail.com>
Reply-To: "general@incubator.apache.org" <general@incubator.apache.org>
Date: Thursday, November 20, 2014 at 3:02 AM
To: "general@incubator.apache.org" <general@incubator.apache.org>
Subject: [PROPOSAL] NiFi for Incubation

>Hello,
>
>
>I would like to propose NiFi as an Apache Incubator Project.
>
>In addition to the copy provided below the Wiki version of the
>proposal can be found here:
>http://wiki.apache.org/incubator/NiFiProposal
>
>Thanks
>
>Joe
>
>
>= NiFi Proposal =
>
>== Abstract ==
>NiFi is a dataflow system based on the concepts of flow-based programming.
>
>== Proposal ==
>NiFi supports powerful and scalable directed graphs of data routing,
>transformation, and system mediation logic.  Some of the high-level
>capabilities and objectives of NiFi include:
>  * Web-based user interface for seamless experience between design,
>control, feedback, and monitoring of data flows
>  * Highly configurable along several dimensions of quality of service
>such as loss tolerant versus guaranteed delivery, low latency versus
>high throughput, and priority based queuing
>  * Fine-grained data provenance for all data received, forked,
>joined, cloned, modified, sent, and ultimately dropped as data reaches
>its configured end-state
>  * Component-based extension model along well defined interfaces
>enabling rapid development and effective testing
>
>== Background ==
>Reliable and effective dataflow between systems can be difficult
>whether you're running scripts on a laptop or have a massive
>distributed computing system operated by numerous teams and
>organizations.  As the volume and rate of data grows and as the number
>of systems, protocols, and formats increase and evolve so too does the
>complexity and need for greater insight and agility.  These are the
>dataflow challenges that NiFi was built to tackle.
>
>NiFi is designed in a manner consistent with the core concepts
>described in flow-based programming as originally documented by J.
>Paul Morrison in the 1970s.  This model lends itself well to visual
>diagramming, concurrency, componentization, testing, and reuse.  In
>addition to staying close to the fundamentals of flow-based
>programming, NiFi provides integration system specific features such
>as: guaranteed delivery; back pressure; ability to gracefully handle
>backlogs and data surges; and an operator interface that enables
>on-the-fly data flow generation, modification, and observation.
>
>== Rationale ==
>NiFi provides a reliable, scalable, manageable and accountable
>platform for developers and technical staff to create and evolve
>powerful data flows.  Such a system is useful in many contexts
>including large-scale enterprise integration, interaction with cloud
>services and frameworks, business to business, intra-departmental, and
>inter-departmental flows.  NiFi fits well within the Apache Software
>Foundation (ASF) family as it depends on numerous ASF projects and
>integrates with several others.  We also anticipate developing
>extensions for several other ASF projects such as Cassandra, Kafka,
>and Storm in the near future.
>
>== Initial Goals ==
>  * Ensure all dependencies are compliant with Apache License version
>2.0 and all that all code and documentation artifacts have the correct
>Apache licensing markings and notice.
>  * Establish a formal release process and schedule, allowing for
>dependable release cycles in a manner consistent with the Apache
>development process.
>  * Determine and establish a mechanism, possibly including a
>sub-project construct, that allows for extensions to the core
>application to occur at a pace that differs from the core application
>itself.
>
>== Current Status ==
>=== Meritocracy ===
>An integration platform is only as good as its ability to integrate
>systems in a reliable, timely, and repeatable manner.  The same can be
>said of its ability to attract talent and a variety of perspectives as
>integration systems by their nature are always evolving.  We will
>actively seek help and encourage promotion of influence in the project
>through meritocracy.
>
>=== Community ===
>Over the past several years, NiFi has developed a strong community of
>both developers and operators within the U.S. government.  We look
>forward to helping grow this to a broader base of industries.
>
>=== Core Developers ===
>The initial core developers are employed by the National Security
>Agency and defense contractors.  We will work to grow the community
>among a more diverse set of developers and industries.
>
>=== Alignment ===
>From its inception, NiFi was developed with an open source philosophy
>in mind and with the hopes of eventually being truly open sourced.
>The Apache way is consistent with the approach we have taken to date.
>The ASF clearly provides a mature and effective environment for
>successful development as is evident across the spectrum of well-known
>projects.  Further, NiFi depends on numerous ASF libraries and
>projects including; ActiveMQ, Ant, Commons, Lucene, Hadoop,
>HttpClient, Jakarta and Maven.  We also anticipate extensions and
>dependencies with several more ASF projects, including Accumulo, Avro,
>Casandra, HBase, JClouds, Storm, Kafka, Thrift, Tika, and others.
>This existing alignment with Apache and the desired community makes
>the Apache Incubator a good fit for NiFi.
>
>== Known Risks ==
>=== Orphaned Products ===
>Risk of orphaning is minimal.  The project user and developer base is
>substantial, growing, and there is already extensive operational use
>of NiFi.
>
>=== Inexperience with Open Source ===
>The initial committers to NiFi have limited experience with true open
>source software development.  However, despite the project origins
>being from closed source development we have modelled our behavior and
>community development on The Apache Way to the greatest extent
>possible.  This environment includes widely accessible source code
>repositories, published artifacts, ticket tracking, and extensive
>documentation. We also encourage contributions and frequent debate and
>hold regular, collaborative discussions through e-mail, chat rooms,
>and in-person meet-ups.  We are committed to the ideals of open source
>software and will eagerly seek out mentors and sponsors who can help
>us quickly come up to speed.
>
>=== Homogenous Developers ===
>The initial committers of NiFi come from a limited set of entities
>though we are committed to recruiting and developing additional
>committers from a broad spectrum of industries and backgrounds.
>
>=== Reliance on Salaried Developers ===
>We expect NiFi development to continue on salaried time and through
>volunteer time.  The initial committers are paid by their employers to
>contribute to this project.  We are committed to developing and
>recruiting participation from developers both salaried and
>non-salaried.
>
>=== Relationship with other Apache Projects ===
>As described in the alignment section, NiFi is already heavily
>dependent on other ASF projects and we anticipate further dependence
>and integration with new and emerging projects in the Apache family.
>
>=== An Excessive Fascination with the Apache Brand ===
>We respect the laudable Apache brand and that is certainly a factor in
>the decision to propose NiFi for the Apache Incubator.  We intend to
>provide a great deal of energy and capability to the ASF through this
>project.  We will be sensitive to and respectful of any overuse of the
>Apache brand and ensure our focus remains on how we benefit the Apache
>community.
>
>=== Documentation ===
>At this time there is no NiFi documentation on the web.  However, we
>have extensive documentation included within the application that
>details usage of the many functions.  We will be rapidly expanding the
>available documentation to cover things like installation, developer
>guide, frequently asked questions, best practices, and more.  This
>documentation will be posted to the NiFi wiki at apache.org.
>
>== Initial Source ==
>NiFi has been in active development since late 2006 with contributions
>from dozens of developers and feedback from hundreds of users and
>developers.  The core codebase is written in Java and includes
>detailed Javadocs and feature documentation.
>
>== Source and Intellectual Property Submission ==
>Previously referred to as Niagarafiles, the NiFi code and
>documentation materials will be submitted by the National Security
>Agency.  NiFi has been developed by a mix of government employees and
>private companies under government contract.  Material developed by
>the government employees is in the public domain and no U.S. copyright
>exists in works of the federal government.  For the contractor
>developed material in the initial submission, the U.S. Government has
>sufficient authority to open source per DFARS 252.227-7014.  NSA has
>submitted the Software Grant Agreement and Corporate Contributor
>License Agreement to the Apache Software Foundation.
>
>== External Dependencies ==
>We have at least one dependency on an LGPL library which we will
>promptly address.  Otherwise, we believe all current dependencies are
>compatible with the ASF guidelines.  Our dependency licenses come from
>the following license styles:  Apache v 2.0, BSD, Public Domain,
>Eclipse Public v1, MIT, CDDL v1.
>
>== Cryptography ==
>Consistent with http://www.apache.org/licenses/exports/ we believe
>NiFi is classified as ECCN 5D002.  NiFi doesn't implement any
>cryptographic algorithms but is designed to use algorithms provided by
>Oracle Java Cryptographic Extensions, BouncyCastle, and JCraft, Inc.
>These cryptographic algorithm providers are used to support SSL,
>SSH/SFTP, and the encryption and decryption of sensitive properties.
>In the event that it becomes necessary we will engage with appropriate
>Apache members to ensure we file any necessary paperwork or clarified
>any cryptographic export license concerns.
>
>== Required Resources ==
>=== Mailing Lists ===
>  * user@nifi.incubator.apache.org
>  * dev@nifi.incubator.apache.org
>  * private@nifi.incubator.apache.org
>  * commits@nifi.incubator.apache.org
>
>=== Source Control ===
>NiFi requests use of Git for source control
>(git://git.apache.org/nifi.git).  We request a writeable Git repo for
>NiFi with mirroring to be setup to Github through INFRA.  We request
>sponsor Benson Margulies (bimargulies) to assist with creating the
>INFRA ticket for this.
>
>=== Issue Tracking ===
>JIRA NiFi (NIFI)
>
>=== Initial Committers ===
>  * Brandon Devries <brandon.devries at gmail dot com>
>  * Matt Gilman <matt.c.gilman at gmail dot com>
>  * Tony Kurc <trkurc at gmail dot com>
>  * Mark Payne <markap14 at hotmail dot com>
>  * Adam Taft <adam at adamtaft dot com>
>  * Joseph Witt <joewitt at gmail dot com>
>
>=== Affiliations ===
>  * Brandon Devries (Requitest, Inc.)
>  * Matt Gilman (Raytheon)
>  * Tony Kurc (National Security Agency)
>  * Mark Payne (Sotera Defense Solutions, Inc.)
>  * Adam Taft (Requitest, Inc.)
>  * Joseph Witt (National Security Agency)
>
>== Sponsors ==
>=== Champion ===
>  * Benson Margulies (Basis Technology) <bimargulies at apache dot org>
>
>=== Nominated Mentors ===
>  * Drew Farris (Booz Allen Hamilton) <drew at apache dot org>
>  * Brock Noland (Cloudera) <brock at apache dot org>
>  * Billie Rinaldi (Hortonworks) <billie at apache dot org>
>
>=== Sponsoring Entity ===
>We request the Apache Incubator to sponsor this project.


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message