incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <>
Subject [Incubator Wiki] Trivial Update of "ApexProposal" by AmolKekre
Date Tue, 04 Aug 2015 06:20:50 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "ApexProposal" page has been changed by AmolKekre:

Added Known Risks

  The DataTorrent team has always focused on building a robust end user community of paying
and non-paying customers. We think that the existing community centered around the existing
google groups mailing list should be relatively easy to transform into an Apache-style community
including both users and developers. 
+ === Meritocracy ===
+ Our proposed list of initial committers include the current RTS R&D team, and our existing
customers. This group will form a base for the broader community we will invite to collaborate
on the codebase. We intend to radically expand the initial developer and user community by
running the project in accordance with the "Apache Way". Users and new contributors will be
treated with respect and welcomed. By participating in the community and providing quality
patches/support that move the project forward, they will earn merit. They also will be encouraged
to provide non-code contributions (documentation, events, presentations, community management,
etc.) and will gain merit for doing so. Those with a proven support and quality track record
will be encouraged to become committers.
- == Community ==
+ === Community ===
  If Apex is accepted for incubation, the primary initial goal will be transitioning the core
community towards embracing the Apache Way of project governance. We will solicit major existing
contributors to become committers on the project from the start. It should be noted that the
existing community is already more diverse in many ways than some top-level Apache projects.
We expect that we can encourage even more diversity.
- == Core Developers ==
+ === Core Developers ===
  While a few core developers are skilled in working in openly governed Apache communities,
most of the core developers are currently NOT affiliated with the ASF and would require new
ICLAs before committing to the project. There would also be a learning curve associated with
this on-boarding. Changing current development practices to be more open will be an important
- == Alignment ==
+ === Alignment ===
  The following existing ASF projects provide related functionality as that provided by Apex
and should be considered when reviewing Apex proposal:
  Apache HadoopⓇ is a distributed storage and processing framework for very large datasets
focusing primarily on batch processing for analytic purposes. Apex is a native YARN application.
The Apex and Malhar roadmap includes plans to continue to leverage YARN, and help the YARN
community develop the ability to support long running applications. Apex uses DFS interface
of its core checkpoint/commit. Malhar has a large number of operators that leverage HDFS and
other Apache projects. Our roadmap includes plans to continue to deepen the currently close
integration with HDFS.
@@ -70, +72 @@

  Project Malhar and Apex are aligned to many more Apache projects and other open source projects
as ease of integration with other technologies is one of the primary goals of this project.
These include Apache Solr, ElasticSearch, MongoDB, Aerospike, ZeroMQ, CouchDB, CouchBase,
MemCache, Redis, RabbitMQ, Apache Derby.
+ == Known Risks ==
+ Development has been sponsored mostly by a single company (DataTorrent, Inc.) thus far and
coordinated mainly by the core DataTorrent RTS and Malhar team, with active participation
from our current customers.
+ For the project to fully transition to the Apache Way governance model, development must
shift towards the merit-centric model of growing a community of contributors balanced with
the needs for extreme stability and core implementation coherency.
+ The tools and development practices in place for the DataTorrent RTS and Malhar products
are compatible with the ASF infrastructure and thus we do not anticipate any on-boarding pains.
Migration from the current GitHub repository is also expected to be straightforward.
+ === Orphaned products ===
+ DataTorrent is fully committed to DataTorrent Apex and Malhar and the product will continue
to be based on the Apex project. Moreover, DataTorrent has a vested interest in making Apex
succeed by driving its close integration with sister ASF projects. We expect this to further
reduces the risk of orphaning the product.
+ === Inexperience with Open Source ===
+ DataTorrent has embraced open source software by open sourcing Malhar project under Apache
2.0 license. DataTorrent team includes veterans from Yahoo! Hadoop team. Although some of
the initial committers have not been developers on an entirely open source, community-driven
project, we expect to bring to bear the open development practices of Malhar to Apex project.
Additionally, several ASF veterans agreed to mentor the project and are listed in this proposal.
The project will rely on their guidance and collective wisdom to quickly transition the entire
team of initial committers towards practicing the Apache Way. DataTorrent is also driving
Kafka on Yarn (KOYA) initiative.
+ === Homogeneous Developers ===
+ While most of the initial committers are employed by DataTorrent, we have already seen a
healthy level of interest from our existing customers and partners. We intend to convert that
interest directly into participation and will be investing in activities to recruit additional
committers from other companies.
+ === Reliance on Salaried Developers ===
+ Most of the contributors are paid to work in the Big Data space. While they might wander
from their current employers, they are unlikely to venture far from their core expertises
and thus will continue to be engaged with the project regardless of their current employers.
+ === Relationships with Other Apache Products ===
+ As mentioned in the Alignment section, Apex may consider various degrees of integration
and code exchange with Apache Hadoop (Yarn and HDFS), Apache Kafka, Apache HBase, Apache Flume,
Apache Cassandra, Apache Accumulo, Apache Tez, Apache Hive, Apache Pig, Apache Storm, Apache
Samza, Apache Spark, Apache Slider. Given the success that the DataTorrent RTS product enjoyed,
we expect integration points to be inside and outside the project. We look forward to collaborating
with these communities as well as other communities under the Apache umbrella.
+ === An Excessive Fascination with the Apache Brand ===
+ While we intend to leverage the Apache ‘branding’ when talking to other projects as
testament of our project’s ‘neutrality’, we have no plans for making use of Apache brand
in press releases nor posting billboards advertising acceptance of Apex into Apache Incubator.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message