incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Incubator Wiki] Update of "ConcertedProposal" by RomanShaposhnik
Date Thu, 10 Sep 2015 16:31:45 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "ConcertedProposal" page has been changed by RomanShaposhnik:
https://wiki.apache.org/incubator/ConcertedProposal?action=diff&rev1=21&rev2=22

  
  = Proposal =
  Concerted is built on the principal that a new type of workload is dominating the scene
and is now needed to be supported. These are the large data set analytical workloads being
analyzed or used on large clusters or high power machines. Large analytical workloads depend
on the ability to query large data sets efficiently and in high concurrency while maintaining
semantics such as immediate consistency. An in memory engine designed to support extreme read
queries while providing support for aggregation through various features (such as multidimensional
representation of tuples) will accelerate many usecases around large scale analytics.
+ 
+ The goal of this proposal is to leverage an existing code base available on Github and licensed
under the Apache License 2.0 to build a community around the project. Currently the community
is small but we have seen a lot of interest from existing ASF projects in helping us achieve
the technological goals.
  
  
  = Background =
@@ -25, +27 @@

      Concerted aims to give expected performance under vast variety of workloads and aims
to have as low footprint as possible.
  
  = Initial Goals =
- The initial goal is to restructure the existing code base so that it becomes easier to include
new contributors and minimize ramp up time, move it to Apache git and build a larger community.
+ The initial goal is to leverage an existing code base and invest in building a community
around the project. We anticipate a lot of initial restructuring of the existing code so that
it becomes easier to include new contributors and minimize ramp up time. We plan to approach
this refactoring in a fully transparent, community-driven way thus starting to practice the
"Apache Way" governance model from the get go.
  
  = Current Status =
- Concerted is currently under restructuring to suit the needs of an open source project.
Current source is available at https://github.com/atris/Concerted (Please note that updated
codebase is not yet present on github) Concerted is currently being licensed under Apache
License 2.0
+ Concerted is currently under restructuring to suit the needs of an open source project.
Current source is available at https://github.com/atris/Concerted (Please note that updated
codebase is not yet present on github) Concerted is currently being licensed under Apache
License 2.0. Most of the code base is implemented in C and C++ and has little external dependencies.
  
  == Meritocracy ==
  
- We will discuss the milestone and the future plan in an open forum. We plan to encourage
an environment that supports a meritocracy.
+ We plan to drive the technical roadmap and implementation in a fully transparent, community-driven
way soliciting feedback from all of the community members and building a consensus-driven
approach to evolving the code base and the community itself. Users and new contributors will
be treated with respect and welcomed. By participating in the community and providing quality
patches/support that move the project forward, contributors will earn merit. They also will
be encouraged to provide non-code contributions (documentation, events, community management,
etc.) and will gain merit for doing so. Those with a proven support and quality track record
will be encouraged to become committers.
  
  == Community ==
  In memory is the new cutting edge thing and a new community around performance oriented
systems and enhancing relational database performance by having complete in memory OLTP engines
will greatly benefit performance. So we expect data warehousing projects and communities as
well as projects and companies looking for high performance OLTP performance. In addition,
Ingenium Data Systems is building products around Concerted and will have salaried developers
contribute to the project as part of job responsibility.
@@ -48, +50 @@

   * Michael Down <michael AT dowuk DOT com>
  
  == Alignment ==
- Concerted will be helpful to systems like Tajo which can benefit with in memory structures
optimized for heavy reads and joins (dimension tables). In addition Concerted will benefit
projects looking for in memory relational database as a metadata store, which is the case
for most of the Apache Big Data projects. We expect Apache Hive, Apache Storm, Apache Tajo
to be utilizing Concerted as a supporting engine. For eg, a data warehouse built on Hive or
Tajo can utilize Concerted as an in memory engine for querying and joining dimensional tables.
+ Concerted will be helpful to systems like Tajo which can benefit with in memory structures
optimized for heavy reads and joins (dimension tables). In addition Concerted will benefit
projects looking for in memory relational database as a metadata store, which is the case
for most of the Apache Big Data projects. We expect Apache HAWQ (incubating), Apache Hive,
Apache Storm, Apache Tajo to be utilizing Concerted as a supporting engine. For eg, a data
warehouse built on HAWQ, Hive or Tajo can utilize Concerted as an in memory engine for querying
and joining dimensional tables.
  
  = Known Risks =
  
@@ -67, +69 @@

  == Relationships with Other Apache Products ==
  Concerted has some overlapping function with Apache Incubator Geode. However, Geode is an
in memory key value store whereas Concerted is a write less read many engine. Concerted will
complement Geode and increase the use cases Geode can support with Concerted's help.
  
- A major objective for Concerted is supporting OLAP workloads and data warehouses with in
memory performance and highly performant reads and joins. Concerted will be collaborating
with many open source projects such as Apache Hive, Apache Tajo etc to support their OLAP
workloads hence enabling them to support larger set of usecases with a better throughput.
For eg, a star schema in Hive will benefit from having dimension tables in Concerted with
highly efficient and scalable reads and joins will be very fast. Similar workload for Tajo.
+ A major objective for Concerted is supporting OLAP workloads and data warehouses with in
memory performance and highly performant reads and joins. Concerted will be collaborating
with many open source projects such as Apache HAWQ (incubating), Apache Hive, Apache Tajo
etc to support their OLAP workloads hence enabling them to support larger set of usecases
with a better throughput. For eg, a star schema in Hive will benefit from having dimension
tables in Concerted with highly efficient and scalable reads and joins will be very fast.
Similar workload for Tajo.
  
  Concerted will fit in many other use cases in Apache spectrum as well. For eg, Concerted
can be used with Apache Geode for in memory aggregation indexing. Concerted can also be used
with Apache Flink for streaming real time data into in memory, perform in memory aggregation
and then performing batch processing for efficiency.
  
  
  == A Excessive Fascination with the Apache Brand ==
- We believe that the Apache brand will help us to find contributors and to grow the community.
The community and development process will make this project more stable and help establish
ubiquitous APIs. In addition, Concerted is looking to support multiple Apache projects in
their use cases and accelerate their performance while soliciting their support in development
of the project. We will not be using Apache brand for excessive branding or with any commercial
aspects of Concerted. Apache brand will primarily be used for community building.
+ We believe that the "Apache Way" governance model will provide additional help to us in
finding contributors and growing the community. The community and development process will
make this project more stable and help establish ubiquitous APIs. In addition, Concerted is
looking to support multiple Apache projects in their use cases and accelerate their performance
while soliciting their support in development of the project. We will not be using Apache
brand for excessive branding or with any commercial aspects of Concerted. Apache brand will
primarily be used for community building.
  
  = Documentation =
  Public documents are currently in development and will be published soon.
@@ -95, +97 @@

  = Required Resources =
  == Mailling List ==
   *private@concerted.incubator.apache.org (moderated subscriptions)
- 
   *commits@concerted.incubator.apache.org
- 
   *dev@concerted.incubator.apache.org
- 
   *issues@concerted.incubator.apache.org
- 
   *user@concerted.incubator.apache.org
  
  == Git Repository ==
@@ -115, +113 @@

   * Continuous Integration
    * Jenkins
   * Wiki
-   * http://wiki.apache.org/concerted
+   * cwiki.apache.org/confluence/display/CONCERTED
  
  = Initial Committers =
-  * Roman Shaposhnik <roman AT shaposhnik DOT org>
+  * Roman Shaposhnik <rvs AT apache DOT org>
   * Daniel Dai <daijy AT apache DOT org>
   * Jake Farrell <jfarrell AT apache DOT org>
   * Lars Hofhansl <larsh AT apache DOT org>
@@ -154, +152 @@

  
  == Champion ==
  
-  * Roman Shaposhnik (roman AT shaposhnik DOT org)
+  * Roman Shaposhnik (rvs AT apache DOT org)
  
  == Nominated Mentors ==
  
-  * Roman Shaposhnik <roman AT shaposhnik DOT org>
+  * Roman Shaposhnik <rvs AT apache DOT org>
   * Daniel Dai <daijy AT apache DOT org>
   * Jake Farrell <jfarrell AT apache DOT org>
   * Lars Hofhansl <larsh AT apache DOT org>

---------------------------------------------------------------------
To unsubscribe, e-mail: cvs-unsubscribe@incubator.apache.org
For additional commands, e-mail: cvs-help@incubator.apache.org


Mime
View raw message