incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Rusek <dru...@twitter.com.INVALID>
Subject Re: [VOTE] Accept DistributedLog into the Apache Incubator
Date Wed, 22 Jun 2016 15:41:57 GMT
+1

On Tue, Jun 21, 2016 at 9:49 PM, Mahak Patidar <mahakp@gmail.com> wrote:

> +1
> On Jun 21, 2016 7:06 PM, "Josh Elser" <elserj@apache.org> wrote:
>
> > +1 (binding)
> >
> > Good luck!
> >
> > Sijie Guo wrote:
> >
> >> Hello All,
> >>
> >> Following the discussion thread, I would like to call a VOTE on
> accepting
> >> DistributedLog into the Apache Incubator.
> >>
> >> [] +1 Accept DistributedLog into the Apache Incubator
> >> [] +0 Abstain.
> >> [] -1 Do not accept DistributedLog into the Apache Incubator because ...
> >>
> >> This vote will be open for at least 72 hours.
> >>
> >> The proposal follows, you can also access the wiki page:
> >> https://wiki.apache.org/incubator/DistributedLogProposal
> >>
> >> Here is my +1.
> >>
> >> Thanks,
> >> Sijie
> >>
> >> = Abstract =
> >> DistributedLog is a high-performance replicated log service. It offers
> >> durability, replication and strong consistency, which provides a
> >> fundamental building block for building reliable distributed systems,
> e.g
> >> replicated-state-machines, general pub/sub systems, distributed
> databases,
> >> distributed queues and etc.
> >>
> >> See “Building Distributedlog - Twitter’s high performance replicated log
> >> service” for details:
> >>
> >>
> https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-log-service
> >>
> >> = Proposal =
> >> We propose to contribute DistributedLog codebase and associated
> artifacts
> >> (e.g. documentation, web-site content etc.) to the Apache Software
> >> Foundation with the intent of forming a productive, meritocratic and
> open
> >> community around DistributedLog’s continued development, according to
> the
> >> ‘Apache Way’.
> >>
> >> = Background =
> >> Engineers at Twitter began developing DistributedLog in early 2013.
> >> DistributedLog is described in a Twitter engineering blog post and
> >> presented at the Messaging Meetup in Sep 2015. It has been released as
> an
> >> Apache-licensed open-source project on GitHub in May 2016.
> >>
> >> DistributedLog is a high-performance replicated log service, which
> >> provides
> >> simple stream-oriented abstractions over log-segments and offers
> >> durability, replication and strong consistency for building reliable
> >> distributed systems. The features offered by DistributedLog includes:
> >>
> >>   * Simple high-level, stream oriented interface
> >>   * Naming and metadata scheme for managing streams and other entities
> >>   * Log data management policies, include data segmentation and data
> >> retention
> >>   * Fast write pipeline leveraging batching and compression
> >>   * Fast read mechanism leveraging long-poll and read-ahead caching
> >>   * Service tiers supporting writer fan-in and reader fan-out
> >>   * Geo-replicated logs
> >>
> >> DistributedLog’s most important benefit is high-performance with a
> strong
> >> durability guarantee, making it extremely appropriate for running
> >> different
> >> workloads from distributed database journaling to real-time stream
> >> computing. Its modern, layered architecture makes it easy to run the
> >> service tiers in multi-tenant datacenter environments such as Apache
> Mesos
> >> or cloud environments such as EC2.
> >>
> >> = Rationale =
> >> DistributedLog is designed to provide core fundamental features like
> >> high-performance, durability and strong consistency to anyone who is
> >> building reliable distributed systems, in a simple and efficient way.
> >>
> >> We believe that the ASF is the right venue to foster an open-source
> >> community around DistributedLog’s development. We expect that
> >> DistributedLog will benefit from collaboration with related Apache
> >> projects, and under the auspices of the ASF will attract talented
> >> contributors who will push DistributedLog’s development forward at a
> >> faster
> >> pace.
> >>
> >> We believe that the timing is right for DistributedLog’s development to
> >> move to the ASF: DistributedLog has already run in production at Twitter
> >> for 3 years and served various workloads including a distributed
> database
> >> journal, reliable cross datacenter replication, search ingestion,
> >> andgeneral pub/sub messaging. The project is stable. We are excited to
> see
> >> where an ASF-based community can take DistributedLog.
> >>
> >> = Current Status =
> >> DistributedLog is a stable project that has been used in production at
> >> Twitter for 3 years. The source code is public at github.com/twitter,
> >> which
> >> will seed the Apache git repository.
> >>
> >> = Meritocracy =
> >> We understand the central importance of meritocracy to the Apache Way.
> We
> >> will work to establish a welcoming, fair and meritocratic community.
> >> Several companies have already expressed interest in this project, and
> we
> >> intend to invite additional developers to participate. We look forward
> to
> >> growing a rich user and developer community.
> >>
> >> = Community =
> >> There is a large need for a performant replicated log service for
> >> applications such as distributed databases, distributed transactional
> >> systems, replicated-state-machines and pub/sub messaging/queuing. We
> want
> >> to attract more developers to the project, and we believe that the ASF’s
> >> open and meritocratic philosophy will help us with this. We note the
> >> success of other similar projects already part of the ASF, like Kafka.
> >>
> >> = Core Developers =
> >> DistributedLog is actively developed within Twitter. Most of the
> >> developers
> >> are from Twitter. Many of them are committers or PMC members of Apache
> >> BookKeeper. Others aren’t currently affiliated with ASF so they will
> >> require new ICLAs.
> >>
> >> = Alignment =
> >> DistributedLog is related to several other Apache projects:
> >>
> >>   * DistributedLog stores log segments as Ledgers in Apache BookKeeper.
> >>   * DistributedLog uses Apache ZooKeeper for naming and metadata
> >> management
> >> and tracking the ownership of logs.
> >>   * DistributedLog uses Apache Thrift as its RPC and serialization
> >> framework.
> >>   * In the long-term, DistributedLog’s data will be stored in Apache
> >> Hadoop
> >> clusters powered by HDFS filesystem for archives and backup.
> >>
> >> = Known Risks =
> >> == Orphaned Products ==
> >> DistributedLog is used as the fundamental messaging infrastructure at
> >> Twitter. It has been serving production traffic for online database
> >> systems, search ingestion and a general pub/sub system. Twitter remains
> >> committed to developing and supporting the project. Twitter has a strong
> >> track record in standing behind projects that were contributed to the
> ASF
> >> by its employees, including Apache Mesos, Apache Aurora, Apache
> >> BookKeeper,
> >> Apache Hadoop. There are many companies are interested in using it in
> >> production.
> >>
> >> == Inexperience with Open Source ==
> >> The core developers of DistributedLog are committers of Apache
> BookKeeper.
> >> Although other committers on the initial list are committers or have
> less
> >> experience with the ASF, they already are active in Apache BookKeeper
> >> community. We are confident that the project can be run in accordance
> with
> >> Apache principles on an ongoing basis.
> >>
> >> == Homogeneous Developers ==
> >> The initial committers are from Twitter. We hope to encourage
> >> contributions
> >> from other developers and grow them into committers after they have had
> >> time to continue their contributions.
> >>
> >> == Reliance on Salaried Developers ==
> >> Many of DistributedLog’s initial set of committers work full-time on
> >> DistributedLog, and are paid to do so. However, as mentioned elsewhere,
> we
> >> anticipate growth in the developer community which we hope will include
> >> people from industry, hobbyists, and academics who have an interested in
> >> distributed messaging systems.
> >>
> >> == Relationships with Other Apache Products ==
> >> DistributedLog uses Apache BookKeeper to store log segments and Apache
> >> ZooKeeper to store log metadata and manage log namespaces. It provides
> an
> >> end-to-end solution for replicated logs, to make building reliable
> >> distributed systems much easier. Unlike Kafka or ActiveMQ,
> DistributedLog
> >> is not a full-fledged pub/sub, queuing or messaging system.  Instead, it
> >> is
> >> targeting on providing a fundamental building block for other
> distributed
> >> systems, offering durability, replication and consistency. So it could
> be
> >> used by other distributed systems, such as transactional log for
> >> replicated
> >> state machines (e.g., HDFS NameNode), WAL for distributed databases
> (e.g.
> >> HBase), Journal for in-memory services (e.g., Kestrel) and even storage
> >> backend for a full-fledged messaging system.
> >>
> >> == An Excessive Fascination with the Apache Brand ==
> >> DistributedLog builds on two existing top-level projects, Apache
> >> BookKeeper
> >> and Apache ZooKeeper. Some of the core developers actively participate
> in
> >> both projects and understand well the implications of being hosted by
> >> Apache. We would like this project to build on the same core values of
> ASF
> >> and to grow a community based on meritocracy. Also, there are several
> >> other
> >> projects already hosted by ASF in this space of reliable messaging and
> >> that
> >> overlap with DistributedLog in interests and scope. Consequently, the
> >> combination of all these observations makes us believe that
> DistributedLog
> >> should be hosted by the ASF.
> >>
> >> = Documentation =
> >> Building DistributedLog: Twitter’s high performance replicated log
> >> service (
> >>
> >>
> https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-log-service
> >> )
> >>
> >> Documentation located in http://distributedlog.io.
> >>
> >> = Initial Source =
> >> DistributedLog’s initial source contribution will come from
> >> http://github.com/twitter/distributedlog/.
> >>
> >> = External Dependencies =
> >> DistributedLog depends upon a number of third-party libraries, which we
> >> list below.
> >>
> >>   * Apache BookKeeper (Apache Software License v2.0)
> >>   * Apache Commons (Apache Software License v2.0)
> >>   * Apache Maven (Apache Software License v2.0)
> >>   * Apache Thrift (Apache Software License v2.0)
> >>   * Apache ZooKeeper (Apache Software License v2.0)
> >>   * Google Guava (Apache Software License v2.0)
> >>   * Mockito (MIT License)
> >>   * Junit (Eclipse Public License 1.0)
> >>   * LZ4-java (Apache Software License v2.0)
> >>   * SLF4J (MIT License)
> >>   * Twitter Finagle (Apache Software License v2.0)
> >>   * Twitter Scrooge (Apache Software License v2.0)
> >>   * Twitter Util (Apache Software License v2.0)
> >>
> >> = Required Resources =
> >> We request that following resources be created for the project to use:
> >>
> >> == Mailing lists ==
> >>   * private@distributedlog.incubator.apache.org (moderated
> subscriptions)
> >>   * commits@distributedlog.incubator.apache.org
> >>   * dev@distributedlog.incubator.apache.org
> >>   * user@distributedlog.incubator.apache.org
> >>
> >> == Git repository ==
> >> https://git.apache.org/distributedlog.git
> >>
> >> == JIRA instance ==
> >> JIRA project DLOG (DLOG or DL)
> >>
> >> = Initial Committers =
> >>   * Sijie Guo (Apache BookKeeper Committer, Twitter)
> >>   * Robin Dhamankar (Apache BookKeeper Committer)
> >>   * Leigh Stewart (Twitter)
> >>   * Dave Rusek (Twitter)
> >>   * Honggang Zhang (Twitter)
> >>   * Jordan Bull (Twitter)
> >>   * Satish Kotha (Twitter)
> >>   * Aniruddha Laud
> >>   * Franck Cuny (Twitter)
> >>   * Eitan Adler (Twitter)
> >>
> >> == Affiliations ==
> >> Most of the initial committers are employees of Twitter, except Robin
> >> Dhamankar and Aniruddha Laud.
> >>
> >> = Sponsors =
> >> == Champion ==
> >> Flavio Junqueira
> >>
> >> == Nominated Mentors ==
> >>   * Flavio Junqueira
> >>   * Chris Nauroth
> >>   * Henry Saputra
> >>
> >> = Sponsoring Entity =
> >> We ask that the Apache Incubator PMC to sponsor this proposal.
> >>
> >>
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message