incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Katherine Marsden <>
Subject [VOTE] Accept Quarks into the Apache Incubator
Date Wed, 24 Feb 2016 17:01:59 GMT
The Quarks proposal has been discussed on the incubator list.  The 
discussion thread is at:

Feedback from the discussion including addition of mentor Justin Mclean 
has been incorporated into the proposal below and available on the wiki at:

Please cast your vote to:
[] +1 - accept Quarks as a new incubating project
[]  0 - not sure
[] -1 - do not accept the Quarks project (because: ...)


Kathey Marsden

= Quarks Proposal =
=== Abstract ===
Quarks is a stream processing programming model and lightweight runtime 
to execute analytics at devices on the edge or at the gateway.

=== Proposal ===
  . Quarks  is a programming model and runtime for streaming analytics 
at the edge.   Applications are developed using a functional flow api 
to  define operations on data streams that is executed as a graph of 
"oplets" in a lightweight embeddable runtime.   The SDK provides 
capabilities like windowing, aggregation  and connectors with an 
extensible model for the community to expand its  capabilities.

=== Background ===
  . Stream processing systems are commonly used to process  data from 
edge devices and there is a need to push some of the  streaming 
analytics to the edge to reduce communication costs, react  locally and 
offload processing from the central systems.  Quarks was developed by 
IBM as an entirely new project to provide an SDK  and lightweight 
embeddable runtime for streaming analytics at the edge.   Quarks was 
created to be an open source project that could provide edge  analytics 
to a broad community and foster collaboration on common  analytics and 
connectors across a broad ecosystem of devices.

=== Rationale ===
  . With the growth in number of connected devices (Internet of Things)  
there is a need to execute analytics at the edge in order to take local  
actions based upon sensor information and/or reduce the volume of data  
sent to back-end analytic systems to reduce communication cost.
  Quarks rationale is to provide  consistent and easy to use programming 
models to allow application developers to focus on their  application 
rather than issues like device connectivity, threading etc.   Quarks' 
functional data flow programming model is similar to systems  like 
Apache Flink, Beam (An incubating Apache project), Java 8 Streams & 
Apache Spark.   The API currently has language bindings for Java8, Java7 
and Android. Quarks was developed to address requirements for analytics 
at the  edge for IoT use cases that were not addressed by central 
analytic  solutions.  We believe that these capabilities will be useful 
to many  organizations and that the diverse nature of edge devices and 
use cases  is best addressed by an open community. Therefore, we would 
like to contribute Quarks to the ASF as an open source project and begin 
developing a community of developers and users within Apache.

=== Initial Goals ===
  . Quarks initial code contribution provides:

  * APIs for developing applications that execute  analytics using a 
per-event (data item) streaming paradigm including  support for windows  
against a stream for aggregation
  * A micro-kernel style runtime for execution.
  * Connectors for MQTT, HTTP, JDBC, File, Apache Kafka &  IBM Watson 
IoT Platform
  * Simple analytics aimed at device sensors (using Apache Common Math)
  * Development mode including a web-console to view the graph of 
running applications
  * Testing mechanism for Quarks applications that integrates with 
assertion based testing systems like JUnit
  * Android specific functionality such as producing a stream that 
contains a phone's sensor events (e.g. ambient temperature, pressure)
  * JUnit tests
  . All of the initial code is implemented using Java 8 and when built 
produces jars that can execute on Java 8, Java 7 and Android. The goal 
is to encourage community contributions in any area of Quarks, to  
expand the community (including new committers) and use of Quarks. We  
expect contributions will be driven by real-world use of Quarks by  
anyone active in the IoT space such as auto manufactures, insurance  
companies, etc. as well as individuals experimenting with devices such  
as Raspberry Pis, Arduinos and/or smart phone apps etc. Contributions 
would be welcomed in any aspect of Quarks including:
  * Support for additional programming languages used in devices such as 
C, OpenSwift, Python etc.
  * Specific device feature (e.g. Raspberry Pi, Android) or protocol 
(e.g. OBD-2) support
  * Connectors for device to device (e.g. AllJoyn), device local data 
sources,  or to back-end systems (e.g. a IoT cloud service)
  * Additional analytics, either exposing more functionality from Apache 
Common Math, other libraries or hand-coded analytics.
  * Improvements to the development console, e.g. additional 
visualizations of running applications
  * Documentation, improving existing documentation or adding new guides 
  * Sample applications
  * Testing
  The  code base has been designed to be modular so that additional 
functionality can be added without having to learn it completely, thus  
new contributors can get involved quickly by initially working on a  
focused item such as an additional analytic or connector. The  only 
constraints on contributions will be to keep Quarks on its focus  of IoT 
and edge computing, with attributes such as small footprint and  
modularity to allow deployments to only include what is needed for that  
specific device and/or application.

=== Current Status ===
  . Quarks is a recently released project on Github    The current code is alpha level code 
but is functional and has some  basic tests.  The team is looking 
forward to working in the Apache  community to enhance the functionality 
to allow robust streaming of  devices on the edge.

==== Meritocracy ====
  . Quarks was originally created by Dan Debrunner, William Marshall,  
Victor Dogaru, Dale LaBossiere and Susan Cline. We plan to embrace  
meritocracy and encourage developers to participate and reach committer  
status.  Dan Debrunner was the initial creator of the Apache Derby code  
and a committer when Derby was accepted into incubation.  He is an  
Apache member and has experience with the Apache Way.  Derby is a   
successful project that embraces the Apache meritocracy and graduated  
from incubation  with a diverse group of committers.
  . With an abundance of devices that potentially can take advantage of  
Quarks, there is a large pool of potential contributors and 
committers.   The initial  team is enthusiastic about assisting and 
encouraging  involvement.

==== Community ====
  . Quarks currently has a very small community as it is new, but our  
goal is to build a diverse community at Apache.  The team strongly  
believes that a diverse and vibrant community is critical as devices on  
the edge vary quite a bit.  The community will benefit from developers  
who have expertise in various devices.   We will seek to build a strong  
developer and user community around Quarks.

==== Core Developers ====
  . The initial developers have many years of development experience in 
stream processing.  The initial development  team includes developers 
who have experience with Apache, including one  Apache member, and with 
other open source projects on Github.

=== Alignment ===
  . Quarks interacts with other Apache solutions such as Apache Kafka 
and  Apache Spark.  Quarks is API driven, modular and written in 100% 
java,  making it easy for developers to pick up and get involved.

=== Known Risks ===
==== Orphaned products ====
  . The contributors are from a leading vendor in this space, who has  
shown a commitment to Apache projects in the past.  They are committed  
to working on the project at least for the next several years, as the  
community grows and becomes more diverse.

==== Inexperience with Open Source ====
  . Several of the core developers have experience with Apache, 
including  a developer who is a committer on Derby and an Apache 
member.  All of  the core developers have some level of experience with 
the use of open  source packages and with contributions on projects on 
sites such as  GitHub.

==== Homogenous Developers ====
  . The initial set of developers come from one company, but we are 
committed to finding a diverse set of committers and contributors. The  
current developers are already very familiar with working with many  
geographies, including developers in most geographies around the 
world.   They  are also very comfortable working in a distributed 

==== Reliance on Salaried Developers ====
  . Quarks currently relies on salaried developers at this time, but we  
expect that Quarks will attract a diverse mix of contributors going  
forward.   For Quarks to fully transition to an "Apache Way" governance 
model, we will embrace the meritocracy-centric way of growing the 
community of contributors.

==== Relationships with Other Apache Products ====
  . These Apache projects are used by the current codebase:
  * Apache Ant - Build
  * Apache Common Math - Initial analytics
  * Apache HTTP Components HttpClient - HTTP connectivity
  * Apache Kafka - Kafka is supported as a message hub between edge 
Quarks applications and back-end analytics systems Events from Quarks 
applications sent through message hubs (such as  Apache Kafka) may be 
consumed by back-end systems such as Apache Flink, Apache Spark, Apache 
Samza, Apache Storm, Beam (in  incubation) or others.

==== A Excessive Fascination with the Apache Brand ====
  . Quarks will benefit greatly from wide collaboration with developers  
working in the device space.  We feel the Apache brand will help 
attract  those developers who really want to contribute to this space.  
Several  developers involved with this project have a very positive 
history with  Derby and feel that Apache is the right place to grow the 
Quarks  community.  We will respect Apache brand policies and follow the 
Apache  way.

=== Documentation ===

=== Initial Source ===
  . Quarks code has been recently released on Github under the Apache 
2.0 license at . It was created by 
a small team of developers, and is written in Java.

=== Source and Intellectual Property Submission Plan ===
  . After acceptance into the incubator, IBM will execute a Software 
Grant Agreement and the source code will be transitioned to the Apache  
infrastructure. The code is already licensed under the Apache Software  
License, version 2.0. We do not know of any legal issues that would  
inhibit the transfer to the ASF.

=== External Dependencies ===
  . The dependencies all have Apache compatible license. These include 
Apache, MIT and EPL. The current dependencies are:
  * D3
  * Jetty
  * Apache Kafka
  * Metrics
  * MQTTV3
  * SLF4J
  * GSON
  * Apache commons Math3
  . Development tools are
  * Java SDK 8
  * Eclipse 4.5
  * Ant 1.9
  * Junit 4.10

=== Cryptography ===
  . No cryptographic code is involved with Quarks.

=== Required Resources ===
==== Mailing lists ====
  . (with moderated subscriptions)

==== Git Repository ====

==== Issue Tracking ====
  . Jira Project Quarks (QUARKS)

==== Other Resources ====
  . Means of setting up regular builds and test cycle.

=== Initial Committers ===
  * Daniel Debrunner:  djd at apache dot org -  CLA on file
  * Susan Cline:  home4slc at pacbell dot net  - CLA on file
  * William Marshall:  wcmarsha at gmail dot com  - CLA on file
  * Victor Dogaru:  vdogaru at gmail dot com - CLA on file
  * Dale LaBossiere:  dml.apache at gmail dot com  - CLA on file

=== Affiliations ===
  * Daniel Debrunner IBM
  * Susan Cline IBM
  * William Marshall IBM
  * Victor Dogaru  IBM
  * Dale Labossiere IBM

=== Additional Interested Contributors ===
  * May Wone: mnwone at gmail dot com
  * Sandeep Deshmukh: sandeep at datatorrent dot com
  * Bhupesh Chawda: bhupeshchawda at gmail dot com

=== Sponsors ===
==== Champion ====
  . Katherine Marsden (kmarsden at apache dot org)

==== Nominated Mentors ====
  * Katherine Marsden (kmarsden at apache dot org)
  * Daniel Debrunner (djd at apache dot org)
  * Luciano Resende (lresende at apache dot org)
  * Justin Mclean (justin at classsoftware dot com)

==== Sponsoring Entity ====
  . The Incubator

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message