incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
Subject Re: HTrace 4.1 release
Date Fri, 04 Mar 2016 00:58:10 GMT
Hi Colin,

Apologies for late response. Yes, I re-cheked the RC and I am +1 for this.
My VOTE is binding.
Thank you, good to see HTrace releasing again.
Lewis

On Thu, Mar 3, 2016 at 4:40 PM, <general-digest-help@incubator.apache.org>
wrote:

>
> From: "Colin P. McCabe" <cmccabe@apache.org>
> To: general@incubator.apache.org
> Cc:
> Date: Thu, 3 Mar 2016 15:08:50 -0800
> Subject: Re: VOTE: HTrace 4.1 release
> Thanks, Marvin.  I'll send out the announce emails shortly.  And
> thanks for voting, everyone!
>
> best,
> Colin
>
> On Wed, Mar 2, 2016 at 1:00 PM, Marvin Humphrey <marvin@rectangular.com>
> wrote:
> > On Wed, Mar 2, 2016 at 11:45 AM, Colin P. McCabe <cmccabe@apache.org>
> wrote:
> >> Thanks for the comments so far, guys.
> >>
> >> Porting over the votes from the HTrace development list:
> >> Michael Stack: +1
> >> Lewis John McGibbney: +1
> >> Masatake Iwasaki: +1
> >> Elliot Clark: +1
> >> and of course myself, +1.
> >>
> >> It seems like we need another IPMC vote on this; does anyone have time
> >> to take a look?
> >
> > Justin Mclean, Lewis John McGibbney, and Michael Stack are all IPMC
> > members who have cast +1 votes in this thread.  You should be all set!
> >
> > Marvin Humphrey
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
>
>
>
> ---------- Forwarded message ----------
> From: Tsuyoshi Ozawa <ozawa@apache.org>
> To: general@incubator.apache.org
> Cc:
> Date: Fri, 4 Mar 2016 09:28:38 +0900
> Subject: Re: [VOTE] Accept Gearpump into the Apache Incubator
> +1(non-binding)
>
> - Tsuyoshi
>
> On Fri, Mar 4, 2016 at 4:44 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
> > +1 (binding)
> >
> >
> >
> > On Thu, Mar 3, 2016 at 9:30 AM, Hao Chen <hao@apache.org> wrote:
> >
> >> +1 (non-binding)
> >>
> >> Regards,
> >> Hao
> >>
> >> On Thu, Mar 3, 2016 at 9:18 AM, Felix Cheung <felixcheung@apache.org>
> >> wrote:
> >>
> >> > +1 (non-binding)
> >> >
> >> > On Thu, Mar 3, 2016 at 9:03 AM Vasudevan, Ramkrishna S <
> >> > ramkrishna.s.vasudevan@intel.com> wrote:
> >> >
> >> > > +1 (non-binding)
> >> > >
> >> > > Regards
> >> > > Ram
> >> > >
> >> > > -----Original Message-----
> >> > > From: Zheng, Kai [mailto:kai.zheng@intel.com]
> >> > > Sent: Wednesday, March 2, 2016 1:54 PM
> >> > > To: general@incubator.apache.org
> >> > > Subject: RE: [VOTE] Accept Gearpump into the Apache Incubator
> >> > >
> >> > > +1 (non-binding).
> >> > >
> >> > > Regards,
> >> > > Kai
> >> > >
> >> > > -----Original Message-----
> >> > > From: JiangAlan [mailto:whjiang@outlook.com]
> >> > > Sent: Wednesday, March 02, 2016 3:21 PM
> >> > > To: general@incubator.apache.org
> >> > > Subject: RE: [VOTE] Accept Gearpump into the Apache Incubator
> >> > >
> >> > > +1 (non-binding)
> >> > >
> >> > > > From: rxin@databricks.com
> >> > > > Date: Tue, 1 Mar 2016 23:14:30 -0800
> >> > > > Subject: Re: [VOTE] Accept Gearpump into the Apache Incubator
> >> > > > To: general@incubator.apache.org
> >> > > >
> >> > > > +1 (binding)
> >> > > >
> >> > > >
> >> > > > On Tue, Mar 1, 2016 at 5:05 PM, James Taylor <
> jamestaylor@apache.org
> >> >
> >> > > wrote:
> >> > > >
> >> > > > > +1 (binding)
> >> > > > >
> >> > > > > On Tue, Mar 1, 2016 at 4:53 PM, Andrew Purtell <
> >> apurtell@apache.org>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Greetings,
> >> > > > > >
> >> > > > > > The discussion of the Gearpump proposal has concluded. Please
> >> vote
> >> > > > > > to accept Gearpump into the Apache Incubator. I will leave
> this
> >> > > > > > vote open
> >> > > > > for
> >> > > > > > at least the next 72 hours and will aim to close it Monday the
> >> 7th
> >> > > > > > of March, 2016 at midnight PT. Gearpump is a flexible,
> efficient,
> >> > > > > > and
> >> > > > > scalable
> >> > > > > > micro-service based real-time big data streaming engine. The
> text
> >> > > > > > of the proposal is included below and is also available at
> >> > > > > > https://wiki.apache.org/incubator/GearpumpProposal
> >> > > > > >
> >> > > > > > [ ] +1 Accept Gearpump as an Apache Incubator podling.
> >> > > > > > [ ] +0 Abstain.
> >> > > > > > [ ] -1 Don’t accept Gearpump as an Apache Incubator podling
> >> because
> >> > > ...
> >> > > > > >
> >> > > > > > Note that while votes from Incubator PMC members are binding,
> all
> >> > > > > > are
> >> > > > > most
> >> > > > > > definitely welcome to vote!
> >> > > > > >
> >> > > > > > I am +1 (binding).
> >> > > > > >
> >> > > > > > Best regards,
> >> > > > > >
> >> > > > > >    - Andy
> >> > > > > >
> >> > > > > > -----
> >> > > > > >
> >> > > > > > = Gearpump Proposal =
> >> > > > > >
> >> > > > > > === Abstract ===
> >> > > > > > Gearpump is a flexible, efficient and scalable micro-service
> >> based
> >> > > > > > real-time big data streaming engine developed by Intel
> >> Corporation
> >> > > > > > which has been licensed by Intel under the Apache License 2.0.
> >> > > > > >
> >> > > > > > === Proposal ===
> >> > > > > > Gearpump is a reactive real-time streaming engine; completely
> >> > > > > > based on
> >> > > > > the
> >> > > > > > micro-service Actor model. Gearpump provides extremely high
> >> > > > > > performance stream processing while maintaining millisecond
> >> latency
> >> > > message delivery.
> >> > > > > > It enables reusable, composable flows or partial graphs that
> can
> >> > > > > > be remotely deployed and executed in a diverse set of
> >> > > > > > environments,
> >> > > > > including
> >> > > > > > IoT edge devices. These flows may be deployed and modified at
> >> > > > > > runtime --
> >> > > > > a
> >> > > > > > capability few real time streaming frameworks provide today.
> >> > > > > >
> >> > > > > > The goal of this proposal is to incubate Gearpump as an Apache
> >> > > > > > project in order to build a diverse, healthy, and
> self-governed
> >> > > > > > open source
> >> > > > > community
> >> > > > > > around this project.
> >> > > > > >
> >> > > > > > === Background ===
> >> > > > > > In past decade, there have been many advances within real-time
> >> > > > > > streaming frameworks. Despite many advances, users of
> streaming
> >> > > > > > frameworks often complain about flexibility, efficiency, and
> >> > > > > > scalability. Gearpump
> >> > > > > endeavors
> >> > > > > > to solve these challenges by adopting the micro-service Actor
> >> > > > > > model. The Actor model was proposed by Carl Hewitt in 1973. In
> >> the
> >> > > > > > Actor model, each actor is a message driven micro-service;
> actors
> >> > > > > > are the basic building blocks of concurrent computation. By
> >> > > > > > leveraging Actor Model’s location transparency
> feature,Gearpump
> >> > > > > > allows a graph to be composed of several partial graphs,
> where,
> >> > > > > > for example, some parts may be deployed to remote IoT edge
> >> > > > > > devices, and other parts to a data center. This division and
> >> > > > > > deployment model can be changed at runtime to adapt to a
> changing
> >> > > > > physical
> >> > > > > > environment, providing extreme flexibility and elasticity in
> >> > > > > > solving various ingestion and analytics problems. We’ve found
> >> > > > > > Actors to be a much smaller computation unit compared with
> >> > > > > > threads, where smaller usually
> >> > > > > means
> >> > > > > > better concurrency, and potentially better CPU utilization.
> >> > > > > >
> >> > > > > > === Rationale ===
> >> > > > > > Gearpump tightly integrates and enhances the big data
> community
> >> of
> >> > > > > > Apache projects. Intel believes Gearpump can bring benefits to
> >> the
> >> > > > > > Apache community in a number of ways:
> >> > > > > >
> >> > > > > > 1. Gearpump complements many existing Apache projects, in
> >> > > > > > particular,
> >> > > > > those
> >> > > > > > commonly found within the big data space. Users of this
> project
> >> > > > > > are also users of other Apache projects, such as Hadoop
> ecosystem
> >> > > > > > projects. It is beneficial to align these projects under the
> ASF
> >> > > > > > umbrella. In real-time streaming, Gearpump offers some special
> >> > > > > > features that are useful for
> >> > > > > Apache
> >> > > > > > users, such as exactly-once processing with millisecond
> message
> >> > > > > > level latency and dynamic DAGs that allow online topology
> >> > > modifications.
> >> > > > > >
> >> > > > > > 2. Gearpump tightly integrates with Apache big data projects.
> It
> >> > > > > > supports for Apache HDFS, YARN, Kafka, and HBase. It uses
> Apache
> >> > > > > > YARN for resource scheduling and Apache HDFS as the essential
> >> > > distributed storage system.
> >> > > > > >
> >> > > > > > 3. The micro-service model of reusable flows that Gearpump has
> >> > > > > > adopted is very unique, and it may become common in the
> >> > > > > > future.Gearpump sets a good example about how distributed
> >> software
> >> > > > > > can be implemented within a micro-service model.  An open
> project
> >> > is
> >> > > of best interest to our users.
> >> > > > > By
> >> > > > > > joining Apache, it will be a neutral infrastructure platform
> that
> >> > > > > > will benefit everyone.
> >> > > > > >
> >> > > > > > 4. The process and development philosophy of Apache will help
> >> > > > > > Gearpump grow, and build a diverse, healthy, and self-governed
> >> > > > > > open source community.
> >> > > > > >
> >> > > > > > === Initial Goals ===
> >> > > > > > 1. Migrate the existing codebase to Apache.
> >> > > > > >
> >> > > > > > 2. Setup Jira, website and other development tools by
> following
> >> > > > > > Apache
> >> > > > > best
> >> > > > > > practices.
> >> > > > > >
> >> > > > > > 3. Start the first release per Apache guidelines as soon as
> >> > possible.
> >> > > > > >
> >> > > > > > === Current Status ===
> >> > > > > > Gearpump is hosted on Github. It has 1922 commits, 38284 line
> of
> >> > > > > > code,
> >> > > > > and
> >> > > > > > 31 major or minor releases, with release notes highlighting
> the
> >> > > > > > changes
> >> > > > > for
> >> > > > > > every release. It is licensed under Apache License Version 2.
> >> > > > > > There is a documentation site at http://gearpump.io
> >> including
> >> > > > > > a user guide, internal details, use cases and a roadmap. There
> >> > > > > is
> >> > > > > > also an issue tracker where every code commit is tracked by a
> bug
> >> > Id.
> >> > > > > Every
> >> > > > > > pull request is reviewed by several reviewers and will only be
> >> > > > > > merged
> >> > > > > based
> >> > > > > > on consensus rule. These match Apache’s development ideals.
> >> > > > > >
> >> > > > > > ==== Meritocracy ====
> >> > > > > > We think an open, fair, and renewing community culture is
> what we
> >> > > > > > need
> >> > > > > and
> >> > > > > > what our users require, that will protect everyone in the
> >> > > > > > community. We would like the project to be free from potential
> >> > > > > > undue influence from any single organization. We will invest
> in
> >> > > supporting a meritocratic model.
> >> > > > > >
> >> > > > > > ==== Community ====
> >> > > > > > Gearpump has a growing community with hundreds of stars on
> Github
> >> > > > > > and an active WeChat group with hundreds of subscriptions. We
> >> > > > > > organize regular offline meetup events. These efforts should
> help
> >> > > > > > us to grow the community at Apache.
> >> > > > > >
> >> > > > > > ==== Core Developers ====
> >> > > > > > Most of the initial committers are Intel employees from China,
> >> the
> >> > > > > > US,
> >> > > > > and
> >> > > > > > Poland. We are committed to build a diverse community which
> >> > > > > > involves more companies and individuals.
> >> > > > > >
> >> > > > > > === Alignment ===
> >> > > > > > Gearpump has good alignment with other Apache projects.
> Gearpump
> >> > > > > > is
> >> > > > > tightly
> >> > > > > > integrated with Apache Hadoop ecosystem. It uses Apache YARN
> for
> >> > > > > > resource scheduling and Apache HDFS for storage. The unique
> >> > > > > > streaming processing abilities Gearpump complements other
> Apache
> >> > > > > > big data projects today. We believe there will be a
> synergistic
> >> > > > > > effect by aligning Gearpump under the Apache umbrella.
> >> > > > > >
> >> > > > > > === Known Risks ===
> >> > > > > >
> >> > > > > > ==== Orphaned products ====
> >> > > > > > Intel has a long-term interest in big data and open source
> and a
> >> > > > > > proven record of contributing to Apache projects. The risk of
> >> > > > > > theGearpump
> >> > > > > project
> >> > > > > > being abandoned is very small. Besides, Intel is seeing an
> >> > > > > > increasing interest in Gearpump from different organizations.
> We
> >> > > > > > are committed to
> >> > > > > get
> >> > > > > > more support, adoption, and code contribution from different
> >> > > companies.
> >> > > > > >
> >> > > > > > ==== Inexperience with Open Source ==== Gearpump is an
> existing
> >> > > > > > project under the Apache License, Version 2.0
> >> > > > > with
> >> > > > > > a long history record of open development. Initial committers
> of
> >> > > > > > this project have years of open sourcing contribution
> >> experiences,
> >> > > > > > including code contribution to HDFS, HBase, Storm, YARN,
> Sqoop,
> >> > > > > > and etc. Some of
> >> > > > > the
> >> > > > > > initial committers are also committers to other Apache
> projects.
> >> > > > > >
> >> > > > > > ==== Homogeneous Developers ====
> >> > > > > > The current list of committers includes developers from
> different
> >> > > > > > geographies and time zones; they are able to collaborate
> >> > > > > > effectively in a geographically dispersed environment. We are
> >> > > > > > committed to recruit more committers from different companies
> to
> >> > get
> >> > > a more diverse mixture.
> >> > > > > >
> >> > > > > > ==== Reliance on Salaried Developers ==== Most of our current
> >> > > > > > Gearpump developers are Intel employees who are contributing
> to
> >> > > > > > this project. Our developers are passionate about this project
> >> and
> >> > > > > > spend a lot of their own personal time on the project. We are
> >> > > > > > confident that their interests will remain strong. We are
> >> > > > > > committed to recruiting additional committers from the
> community
> >> as
> >> > > well.
> >> > > > > >
> >> > > > > > ==== Relationships with Other Apache Product ==== Gearpump
> >> > > > > > codebase is closely integrated with Apache Hadoop, Apache
> HBase,
> >> > > > > > and Apache Kafka. Gearpump also has some similarities with
> Apache
> >> > > Storm.
> >> > > > > > Although Gearpump and Storm are both systems for real-time
> stream
> >> > > > > > processing, they have fundamentally different architectures.
> In
> >> > > > > particular,
> >> > > > > > Gearpump adopts the micro-service model, building on the Akka
> >> > > > > > framework, for concurrency, isolation and error handling,
> which
> >> we
> >> > > > > > believe is a
> >> > > > > future
> >> > > > > > trend for building distributed software. We look forward to
> >> > > > > > collaboration with other Apache communities.
> >> > > > > >
> >> > > > > > ==== An Excessive Fascination with the Apache Brand ==== The
> ASF
> >> > > > > > has a strong brand; we appreciate that fact and will protect
> the
> >> > > > > > brand. Gearpump is an existing open source project with many
> >> > > > > > committers
> >> > > > > and
> >> > > > > > years of effort.  The reasons to join Apache are outlined in
> the
> >> > > > > Rationale
> >> > > > > > section above.
> >> > > > > >
> >> > > > > > === Documentation ===
> >> > > > > > Information on Gearpump can be found at:
> >> > > > > > Gearpump website: http://gearpump.io
> >> > > > > > Codebase: https://github.com/gearpump/gearpump
> >> > > > > >
> >> > > > > > === Initial Source and Intellectual Property Submission Plan
> ===
> >> > > > > > The Gearpump codebase is currently hosted on Github:
> >> > > > > > https://github.com/ gearpump/gearpump. We will use this
> codebase
> >> > > > > > to migrate to the Apache foundation. The Gearpump source code
> is
> >> > > > > > licensed under Apache License Version 2.0 and will be kept
> that
> >> > > > > > way. All contributions on the project will be licensed
> directly
> >> to
> >> > > > > > the Apache foundation through signed Individual Contributor
> >> > > > > > License Agreements or Corporate Contributor
> >> > > > > License
> >> > > > > > Agreements.
> >> > > > > >
> >> > > > > > === External Dependencies ===
> >> > > > > > All of Gearpump dependencies are distributed under Apache
> >> > > > > > compatible licenses.
> >> > > > > >
> >> > > > > > Gearpump leverages Akka which has Apache 2.0 licensing for
> >> current
> >> > > > > > and planned versions
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> http://doc.akka.io/docs/akka/2.3.12/project/licenses.html#Licenses_f
> >> > > > > or_Dependency_Libraries
> >> > > > > >
> >> > > > > > === Cryptography ===
> >> > > > > > Gearpump does not include or utilize cryptographic code.
> >> > > > > >
> >> > > > > > === Required Resources ===
> >> > > > > > We request that following resources be created for the
> project to
> >> > > > > > use
> >> > > > > >
> >> > > > > > ==== Mailing lists ====
> >> > > > > >
> >> > > > > > gearpump-private@incubator.apache.org (with moderated
> >> > > > > > subscriptions) gearpump-dev gearpump-user gearpump-commits
> >> > > > > >
> >> > > > > > ==== Git repository ====
> >> > > > > > Git is the preferred source control system: git://
> >> > > > > git.apache.org/gearpump
> >> > > > > >
> >> > > > > > ==== Documentation ====
> >> > > > > > https://gearpump.incubator.apache.org/docs/
> >> > > > > >
> >> > > > > > ==== JIRA instance ====
> >> > > > > > JIRA Gearpump (GEARPUMP)
> >> > > > > > https://issues.apache.org/jira/browse/gearpump
> >> > > > > >
> >> > > > > > === Initial Committers ===
> >> > > > > > * Xiang Zhong <xiang dot zhong at intel dot com>
> >> > > > > >
> >> > > > > > * Tianlun Zhang <tianlun dot zhang at intel dot com>
> >> > > > > >
> >> > > > > > * Qian Xu <qian dot a dot xu at intel dot com>
> >> > > > > >
> >> > > > > > * Huafeng Wang <huafeng dot wang at intel dot com>
> >> > > > > >
> >> > > > > > * Kam Kasravi <kam dot d dot kasravi at intel dot com>
> >> > > > > >
> >> > > > > > * Weihua Jiang <weihua dot jiang at intel dot com>
> >> > > > > >
> >> > > > > > * Tomasz Targonski <tomasz dot targonski at intel dot com>
> >> > > > > >
> >> > > > > > * Karol Brejna <karol dot brejna at intel dot com>
> >> > > > > >
> >> > > > > > * Gang Wang <gang1 dot wang at intel dot com>
> >> > > > > >
> >> > > > > > * Mark Chmarny <mark dot chmarny at intel dot com>
> >> > > > > >
> >> > > > > > * Xinglang Wang <xingwang at ebay dot com >
> >> > > > > >
> >> > > > > > * Lan Wang <lan dot wanglan at huawei dot com>
> >> > > > > >
> >> > > > > > * Jianzhong Chen <jianzhong dot chen at cloudera dot com>
> >> > > > > >
> >> > > > > > * Xuefu Zhang <xuefu at apache dot org>
> >> > > > > >
> >> > > > > > * Rui Li <rui dot li at intel dot com>
> >> > > > > >
> >> > > > > > === Affiliations ===
> >> > > > > > * Xiang Zhong –  Intel
> >> > > > > >
> >> > > > > > * Tianlun Zhang –  Intel
> >> > > > > >
> >> > > > > > * Qian Xu –  Intel
> >> > > > > >
> >> > > > > > * Huafeng Wang –  Intel
> >> > > > > >
> >> > > > > > * Kam Kasravi –  Intel
> >> > > > > >
> >> > > > > > * Weihua Jiang –  Intel
> >> > > > > >
> >> > > > > > * Tomasz Targonski – Intel
> >> > > > > >
> >> > > > > > * Karol Brejna – Intel
> >> > > > > >
> >> > > > > > * Mark Chmarny – Intel
> >> > > > > >
> >> > > > > > * Gang Wang – Intel
> >> > > > > >
> >> > > > > > * Mark Chmarny  – Intel
> >> > > > > >
> >> > > > > > * Xinglang Wang  – Ebay
> >> > > > > >
> >> > > > > > * Lan Wang – Huawei
> >> > > > > >
> >> > > > > > * Jianzhong Chen – Cloudera
> >> > > > > >
> >> > > > > > * Xuefu Zhang – Cloudera
> >> > > > > >
> >> > > > > > * Rui Li  – Intel
> >> > > > > >
> >> > > > > > === Sponsors ===
> >> > > > > >
> >> > > > > > ==== Champion ====
> >> > > > > > Andrew Purtell <apurtell at apache dot org>
> >> > > > > >
> >> > > > > > ==== Nominated Mentors ====
> >> > > > > > * Andrew Purtell <apurtell at apache dot org>
> >> > > > > >
> >> > > > > > * Jarek Jarcec Cecho <Jarcec at cloudera dot com>
> >> > > > > >
> >> > > > > > * Todd Lipcon <todd at cloudera dot com>
> >> > > > > >
> >> > > > > > * Xuefu Zhang <xuefu at apache dot org>
> >> > > > > >
> >> > > > > > * Reynold Xin <rxin at databricks dot com>
> >> > > > > >
> >> > > > > > ==== Sponsoring Entity ====
> >> > > > > > Apache Incubator PMC
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> >> > > For additional commands, e-mail: general-help@incubator.apache.org
> >> > >
> >> >
> >>
>
>
>
> ---------- Forwarded message ----------
> From: "Cheng, Hao" <hao.cheng@intel.com>
> To: "general@incubator.apache.org" <general@incubator.apache.org>
> Cc:
> Date: Fri, 4 Mar 2016 00:40:33 +0000
> Subject: RE: [VOTE] Accept Mnemonic into the Apache Incubator
> +1 (binding)
>
> Hao
>
> -----Original Message-----
> From: Ted Dunning [mailto:ted.dunning@gmail.com]
> Sent: Friday, March 4, 2016 5:32 AM
> To: general@incubator.apache.org
> Subject: Re: [VOTE] Accept Mnemonic into the Apache Incubator
>
> I thought I sent a vote in earlier today but I may have hit the wrong
> thread.
>
> Please accept this vote.
>
> +1 (binding)
>
>
>
> On Thu, Mar 3, 2016 at 11:19 AM, P. Taylor Goetz <ptgoetz@gmail.com>
> wrote:
>
> > +1 (binding)
> >
> > -Taylor
> >
> > > On Feb 29, 2016, at 12:37 PM, Patrick Hunt <phunt@apache.org> wrote:
> > >
> > > Hi folks,
> > >
> > > OK the discussion is now completed. Please VOTE to accept Mnemonic
> > > into the Apache Incubator. I’ll leave the VOTE open for at least the
> > > next 72 hours, with hopes to close it Thursday the 3rd of March,
> > > 2016 at 10am PT.
> > > https://wiki.apache.org/incubator/MnemonicProposal
> > >
> > > [ ] +1 Accept Mnemonic as an Apache Incubator podling.
> > > [ ] +0 Abstain.
> > > [ ] -1 Don’t accept Mnemonic as an Apache Incubator podling because..
> > >
> > > Of course, I am +1 on this. Please note VOTEs from Incubator PMC
> > > members are binding but all are welcome to VOTE!
> > >
> > > Regards,
> > >
> > > Patrick
> > >
> > > --------------------
> > > = Mnemonic Proposal =
> > > === Abstract ===
> > > Mnemonic is a Java based non-volatile memory library for in-place
> > > structured data processing and computing. It is a solution for
> > > generic object and block persistence on heterogeneous block and
> > > byte-addressable devices, such as DRAM, persistent memory, NVMe,
> > > SSD, and cloud network storage.
> > >
> > > === Proposal ===
> > > Mnemonic is a structured data persistence in-memory in-place library
> > > for Java-based applications and frameworks. It provides unified
> > > interfaces for data manipulation on heterogeneous
> > > block/byte-addressable devices, such as DRAM, persistent memory,
> > > NVMe, SSD, and cloud network devices.
> > >
> > > The design motivation for this project is to create a non-volatile
> > > programming paradigm for in-memory data object persistence,
> > > in-memory data objects caching, and JNI-less IPC.
> > > Mnemonic simplifies the usage of data object caching, persistence,
> > > and JNI-less IPC for massive object oriented structural datasets.
> > >
> > > Mnemonic defines Non-Volatile Java objects that store data fields in
> > > persistent memory and storage. During the program runtime, only
> > > methods and volatile fields are instantiated in Java heap,
> > > Non-Volatile data fields are directly accessed via GET/SET operation
> > > to and from persistent memory and storage. Mnemonic avoids SerDes
> > > and significantly reduces amount of garbage in Java heap.
> > >
> > > Major features of Mnemonic:
> > > * Provides an abstract level of viewpoint to utilize heterogeneous
> > > block/byte-addressable device as a whole (e.g., DRAM, persistent
> > > memory, NVMe, SSD, HD, cloud network Storage).
> > >
> > > * Provides seamless support object oriented design and programming
> > > without adding burden to transfer object data to different form.
> > >
> > > * Avoids the object data serialization/de-serialization for data
> > > retrieval, caching and storage.
> > >
> > > * Reduces the consumption of on-heap memory and in turn to reduce
> > > and stabilize Java Garbage Collection (GC) pauses for latency
> > > sensitive applications.
> > >
> > > * Overcomes current limitations of Java GC to manage much larger
> > > memory resources for massive dataset processing and computing.
> > >
> > > * Supports the migration data usage model from traditional
> > > NVMe/SSD/HD to non-volatile memory with ease.
> > >
> > > * Uses lazy loading mechanism to avoid unnecessary memory
> > > consumption if some data does not need to use for computing
> immediately.
> > >
> > > * Bypasses JNI call for the interaction between Java runtime
> > > application and its native code.
> > >
> > > * Provides an allocation aware auto-reclaim mechanism to prevent
> > > external memory resource leaking.
> > >
> > >
> > > === Background ===
> > > Big Data and Cloud applications increasingly require both high
> > > throughput and low latency processing. Java-based applications
> > > targeting the Big Data and Cloud space should be tuned for better
> > > throughput, lower latency, and more predictable response time.
> > > Typically, there are some issues that impact BigData applications'
> > > performance and scalability:
> > >
> > > 1) The Complexity of Data Transformation/Organization: In most
> > > cases, during data processing, applications use their own
> > > complicated data caching mechanism for SerDes data objects, spilling
> > > to different storage and eviction large amount of data. Some data
> > > objects contains complex values and structure that will make it much
> > > more difficulty for data organization. To load and then parse/decode
> > > its datasets from storage consumes high system resource and
> computation power.
> > >
> > > 2) Lack of Caching, Burst Temporary Object Creation/Destruction
> > > Causes Frequent Long GC Pauses: Big Data computing/syntax generates
> > > large amount of temporary objects during processing, e.g. lambda,
> > > SerDes, copying and etc. This will trigger frequent long Java GC
> > > pause to scan references, to update references lists, and to copy
> > > live objects from one memory location to another blindly.
> > >
> > > 3) The Unpredictable GC Pause: For latency sensitive applications,
> > > such as database, search engine, web query, real-time/streaming
> > > computing, require latency/request-response under control. But
> > > current Java GC does not provide predictable GC activities with
> > > large on-heap memory management.
> > >
> > > 4) High JNI Invocation Cost: JNI calls are expensive, but high
> > > performance applications usually try to leverage native code to
> > > improve performance, however, JNI calls need to convert Java objects
> > > into something that C/C++ can understand. In addition, some
> > > comprehensive native code needs to communicate with Java based
> > > application that will cause frequently JNI call along with stack
> > > marshalling.
> > >
> > > Mnemonic project provides a solution to address above issues and
> > > performance bottlenecks for structured data processing and computing.
> > > It also simplifies the massive data handling with much reduced GC
> > > activity.
> > >
> > > === Rationale ===
> > > There are strong needs for a cohesive, easy-to-use non-volatile
> > > programing model for unified heterogeneous memory resources
> > > management and allocation. Mnemonic project provides a reusable and
> > > flexible framework to accommodate other special type of memory/block
> > > devices for better performance without changing client code.
> > >
> > > Most of the BigData frameworks (e.g., Apache Spark™, Apache™
> > > Hadoop®, Apache HBase™, Apache Flink™, Apache Kafka™, etc.) have
> > > their own complicated memory management modules for caching and
> > > checkpoint. Many approaches increase the complexity and are
> > > error-prone to maintain code.
> > >
> > > We have observed heavy overheads during the operations of data
> > > parse, SerDes, pack/unpack, code/decode for data loading, storage,
> > > checkpoint, caching, marshal and transferring. Mnemonic provides a
> > > generic in-memory persistence object model to address those
> > > overheads for better performance. In addition, it manages its
> > > in-memory persistence objects and blocks in the way that GC does,
> > > which means their underlying memory resource is able to be reclaimed
> > > without explicitly releasing it.
> > >
> > > Some existing Big Data applications suffer from poor Java GC
> > > behaviors when they process their massive unstructured datasets.
> > > Those behaviors either cause very long stop-the-world GC pauses or
> > > take significant system resources during computing which impact
> > > throughput and incur significant perceivable pauses for interactive
> analytics.
> > >
> > > There are more and more computing intensive Big Data applications
> > > moving down to rely on JNI to offload their computing tasks to
> > > native code which dramatically increases the cost of JNI invocation
> and IPC.
> > > Mnemonic provides a mechanism to communicate with native code
> > > directly through in-place object data update to avoid complex object
> > > data type conversion and stack marshaling. In addition, this project
> > > can be extended to support various lockers for threads between Java
> > > code and native code.
> > >
> > > === Initial Goals ===
> > > Our initial goal is to bring Mnemonic into the ASF and transit the
> > > engineering and governance processes to the "Apache Way."  We would
> > > like to enrich a collaborative development model that closely aligns
> > > with current and future industry memory and storage technologies.
> > >
> > > Another important goal is to encourage efforts to integrate
> > > non-volatile programming model into data centric
> > > processing/analytics frameworks/applications, (e.g., Apache Spark™,
> > > Apache HBase™, Apache Flink™, Apache™ Hadoop®, Apache Cassandra™,
> etc.).
> > >
> > > We expect Mnemonic project to be continuously developing new
> > > functionalities in an open, community-driven way. We envision
> > > accelerating innovation under ASF governance in order to meet the
> > > requirements of a wide variety of use cases for in-memory
> > > non-volatile and volatile data caching programming.
> > >
> > > === Current Status ===
> > > Mnemonic project is available at Intel’s internal repository and
> > > managed by its designers and developers. It is also temporary hosted
> > > at Github for general view
> > > https://github.com/NonVolatileComputing/Mnemonic.git
> > >
> > > We have integrated this project for Apache Spark™ 1.5.0 and get 2X
> > > performance improvement ratio for Spark™ MLlib k-means workload and
> > > observed expected benefits of removing SerDes, reducing total GC
> > > pause time by 40% from our experiments.
> > >
> > > ==== Meritocracy ====
> > > Mnemonic was originally created by Gang (Gary) Wang and Yanping Wang
> > > in early 2015. The initial committers are the current Mnemonic R&D
> > > team members from US, China, and India Big Data Technologies Group
> > > at Intel. This group will form a base for much broader community to
> > > collaborate on this code base.
> > >
> > > We intend to radically expand the initial developer and user
> > > community by running the project in accordance with the "Apache
> > > Way." Users and new contributors will be treated with respect and
> > > welcomed. By participating in the community and providing quality
> > > patches/support that move the project forward, they will earn merit.
> > > They also will be encouraged to provide non-code contributions
> > > (documentation, events, community management, etc.) and will gain
> > > merit for doing so. Those with a proven support and quality track
> > > record will be encouraged to become committers.
> > >
> > > ==== Community ====
> > > If Mnemonic is accepted for incubation, the primary initial goal is
> > > to transit the core community towards embracing the Apache Way of
> > > project governance. We would solicit major existing contributors to
> > > become committers on the project from the start.
> > >
> > > ==== Core Developers ====
> > > Mnemonic core developers are all skilled software developers and
> > > system performance engineers at Intel Corp with years of experiences
> > > in their fields. They have contributed many code to Apache projects.
> > > There are PMCs and experienced committers have been working with us
> > > from Apache Spark™, Apache HBase™, Apache Phoenix™, Apache™ Hadoop®
> > > for this project's open source efforts.
> > >
> > > === Alignment ===
> > > The initial code base is targeted to data centric processing and
> > > analyzing in general. Mnemonic has been building the connection and
> > > integration for Apache projects and other projects.
> > >
> > > We believe Mnemonic will be evolved to become a promising project
> > > for real-time processing, in-memory streaming analytics and more,
> > > along with current and future new server platforms with persistent
> > > memory as base storage devices.
> > >
> > > === Known Risks ===
> > > ==== Orphaned products ====
> > > Intel’s Big Data Technologies Group is actively working with
> > > community on integrating this project to Big Data frameworks and
> applications.
> > > We are continuously adding new concepts and codes to this project
> > > and support new usage cases and features for Apache Big Data ecosystem.
> > >
> > > The project contributors are leading contributors of Hadoop-based
> > > technologies and have a long standing in the Hadoop community. As we
> > > are addressing major Big Data processing performance issues, there
> > > is minimal risk of this work becoming non-strategic and unsupported.
> > >
> > > Our contributors are confident that a larger community will be
> > > formed within the project in a relatively short period of time.
> > >
> > > ==== Inexperience with Open Source ==== This project has long
> > > standing experienced mentors and interested contributors from Apache
> > > Spark™, Apache HBase™, Apache Phoenix™, Apache™ Hadoop® to help us
> > > moving through open source process. We are actively working with
> > > experienced Apache community PMCs and committers to improve our
> > > project and further testing.
> > >
> > > ==== Homogeneous Developers ====
> > > All initial committers and interested contributors are employed at
> > > Intel. As an infrastructure memory project, there are wide range of
> > > Apache projects are interested in innovative memory project to fit
> > > large sized persistent memory and storage devices. Various Apache
> > > projects such as Apache Spark™, Apache HBase™, Apache Phoenix™,
> > > Apache Flink™, Apache Cassandra™ etc. can take good advantage of
> > > this project to overcome serialization/de-serialization, Java GC,
> > > and caching issues. We expect a wide range of interest will be
> > > generated after we open source this project to Apache.
> > >
> > > ==== Reliance on Salaried Developers ==== All developers are paid by
> > > their employers to contribute to this project. We welcome all others
> > > to contribute to this project after it is open sourced.
> > >
> > > ==== Relationships with Other Apache Product ==== Relationship with
> > > Apache™ Arrow:
> > > Arrow's columnar data layout allows great use of CPU caches & SIMD.
> > > It places all data that relevant to a column operation in a compact
> > > format in memory.
> > >
> > > Mnemonic directly puts the whole business object graphs on external
> > > heterogeneous storage media, e.g. off-heap, SSD. It is not necessary
> > > to normalize the structures of object graphs for caching, checkpoint
> > > or storing. It doesn’t require developers to normalize their data
> > > object graphs. Mnemonic applications can avoid indexing & join
> > > datasets compared to traditional approaches.
> > >
> > > Mnemonic can leverage Arrow to transparently re-layout qualified
> > > data objects or create special containers that is able to
> > > efficiently hold those data records in columnar form as one of major
> > > performance optimization constructs.
> > >
> > > Mnemonic can be integrated into various Big Data and Cloud
> > > frameworks and applications.
> > > We are currently working on several Apache projects with Mnemonic:
> > > For Apache Spark™ we are integrating Mnemonic to improve:
> > > a) Local checkpoints
> > > b) Memory management for caching
> > > c) Persistent memory datasets input
> > > d) Non-Volatile RDD operations
> > > The best use case for Apache Spark™ computing is that the input data
> > > is stored in form of Mnemonic native storage to avoid caching its
> > > row data for iterative processing. Moreover, Spark applications can
> > > leverage Mnemonic to perform data transforming in persistent or
> > > non-persistent memory without SerDes.
> > >
> > > For Apache™ Hadoop®, we are integrating HDFS Caching with Mnemonic
> > > instead of mmap. This will take advantage of persistent memory
> > > related features. We also plan to evaluate to integrate in Namenode
> > > Editlog, FSImage persistent data into Mnemonic persistent memory area.
> > >
> > > For Apache HBase™, we are using Mnemonic for BucketCache and
> > > evaluating performance improvements.
> > >
> > > We expect Mnemonic will be further developed and integrated into
> > > many Apache BigData projects and so on, to enhance memory management
> > > solutions for much improved performance and reliability.
> > >
> > > ==== An Excessive Fascination with the Apache Brand ==== While we
> > > expect Apache brand helps to attract more contributors, our
> > > interests in starting this project is based on the factors mentioned
> > > in the Rationale section.
> > >
> > > We would like Mnemonic to become an Apache project to further foster
> > > a healthy community of contributors and consumers in BigData
> > > technology R&D areas. Since Mnemonic can directly benefit many
> > > Apache projects and solves major performance problems, we expect the
> > > Apache Software Foundation to increase interaction with the larger
> community as well.
> > >
> > > === Documentation ===
> > > The documentation is currently available at Intel and will be posted
> > > under: https://mnemonic.incubator.apache.org/docs
> > >
> > > === Initial Source ===
> > > Initial source code is temporary hosted Github for general viewing:
> > > https://github.com/NonVolatileComputing/Mnemonic.git
> > > It will be moved to Apache http://git.apache.org/ after podling.
> > >
> > > The initial Source is written in Java code (88%) and mixed with JNI
> > > C code (11%) and shell script (1%) for underlying native allocation
> > > libraries.
> > >
> > > === Source and Intellectual Property Submission Plan === As soon as
> > > Mnemonic is approved to join the Incubator, the source code will be
> > > transitioned via the Software Grant Agreement onto ASF
> > > infrastructure and in turn made available under the Apache License,
> > > version 2.0.
> > >
> > > === External Dependencies ===
> > > The required external dependencies are all Apache licenses or other
> > > compatible Licenses
> > > Note: The runtime dependent licenses of Mnemonic are all declared as
> > > Apache 2.0, the GNU licensed components are used for Mnemonic build
> > > and deployment. The Mnemonic JNI libraries are built using the GNU
> > > tools.
> > >
> > > maven and its plugins (http://maven.apache.org/ ) [Apache 2.0]
> > > JDK8 or OpenJDK 8 (http://java.com/) [Oracle or Openjdk JDK License]
> > > Nvml (http://pmem.io ) [optional] [Open Source] PMalloc
> > > (https://github.com/bigdata-memory/pmalloc ) [optional] [Apache
> > 2.0]
> > >
> > > Build and test dependencies:
> > > org.testng.testng v6.8.17  (http://testng.org) [Apache 2.0]
> > > org.flowcomputing.commons.commons-resgc v0.8.7 [Apache 2.0]
> > > org.flowcomputing.commons.commons-primitives v.0.6.0 [Apache 2.0]
> > > com.squareup.javapoet v1.3.1-SNAPSHOT [Apache 2.0]
> > > JDK8 or OpenJDK 8 (http://java.com/) [Oracle or Openjdk JDK License]
> > >
> > > === Cryptography ===
> > > Project Mnemonic does not use cryptography itself, however, Hadoop
> > > projects use standard APIs and tools for SSH and SSL communication
> > > where necessary.
> > >
> > > === Required Resources ===
> > > We request that following resources be created for the project to
> > > use
> > >
> > > ==== Mailing lists ====
> > > private@mnemonic.incubator.apache.org (moderated subscriptions)
> > > commits@mnemonic.incubator.apache.org
> > > dev@mnemonic.incubator.apache.org
> > >
> > > ==== Git repository ====
> > > https://github.com/apache/incubator-mnemonic
> > >
> > > ==== Documentation ====
> > > https://mnemonic.incubator.apache.org/docs/
> > >
> > > ==== JIRA instance ====
> > > https://issues.apache.org/jira/browse/mnemonic
> > >
> > > === Initial Committers ===
> > > * Gang (Gary) Wang (gang1 dot wang at intel dot com)
> > >
> > > * Yanping Wang (yanping dot wang at intel dot com)
> > >
> > > * Uma Maheswara Rao G (umamahesh at apache dot org)
> > >
> > > * Kai Zheng (drankye at apache dot org)
> > >
> > > * Rakesh Radhakrishnan Potty  (rakeshr at apache dot org)
> > >
> > > * Sean Zhong  (seanzhong at apache dot org)
> > >
> > > * Henry Saputra  (hsaputra at apache dot org)
> > >
> > > * Hao Cheng (hao dot cheng at intel dot com)
> > >
> > > === Additional Interested Contributors ===
> > > * Debo Dutta (dedutta at cisco dot com)
> > >
> > > * Liang Chen (chenliang613 at Huawei dot com)
> > >
> > > === Affiliations ===
> > > * Gang (Gary) Wang, Intel
> > >
> > > * Yanping Wang, Intel
> > >
> > > * Uma Maheswara Rao G, Intel
> > >
> > > * Kai Zheng, Intel
> > >
> > > * Rakesh Radhakrishnan Potty, Intel
> > >
> > > * Sean Zhong, Intel
> > >
> > > * Henry Saputra, Independent
> > >
> > > * Hao Cheng, Intel
> > >
> > > === Sponsors ===
> > > ==== Champion ====
> > > Patrick Hunt
> > >
> > > ==== Nominated Mentors ====
> > > * Patrick Hunt <phunt at apache dot org> - Apache IPMC member
> > >
> > > * Andrew Purtell <apurtell at apache dot org > - Apache IPMC member
> > >
> > > * James Taylor <jamestaylor at apache dot org> - Apache IPMC member
> > >
> > > * Henry Saputra <hsaputra at apache dot org> - Apache IPMC member
> > >
> > > ==== Sponsoring Entity ====
> > > Apache Incubator PMC
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > For additional commands, e-mail: general-help@incubator.apache.org
> > >
> >
> >
>
>


-- 
*Lewis*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message