incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: [PROPOSAL] Weave for Apache Incubator
Date Tue, 29 Oct 2013 23:54:54 GMT
In which case, maybe consider the related words:

Apache Warp
Apache Weft

Just a thought.

On 29 October 2013 22:14, Upayavira <uv@odoko.co.uk> wrote:
> And Apache Wave too (which is what I first saw before I read the title
> more carefully).
>
> Upayavira
>
> On Tue, Oct 29, 2013, at 09:12 PM, Matt Benson wrote:
>> Hi,
>>   I am concerned about potential confusion with Apache Commons Weaver
>>   [1].
>>
>> Matt
>>
>> [1] https://commons.apache.org/proper/commons-weaver/
>>
>>
>> On Tue, Oct 29, 2013 at 2:53 PM, Andreas Neumann <anew@apache.org> wrote:
>>
>> > I would like to propose Weave, an abstraction over Apache Hadoop® YARN to
>> > reduce the complexity of developing distributed applications, as an
>> > Apache Incubator
>> > podling.
>> >
>> > The proposal is included in plain text. I would also like to put this on
>> > the wiki, but I appear to lack privileges to create pages. What do I need
>> > to do to get permission?
>> >
>> > -Andreas.
>> >
>> > Abstract
>> > ========
>> >
>> > Weave is an abstraction over Apache Hadoop® YARN that reduces the
>> > complexity of developing distributed applications, allowing developers to
>> > focus more on their business logic.
>> >
>> > Proposal
>> > ========
>> >
>> > Weave is a set of libraries that reduces the complexity of developing
>> > distributed applications. It exposes the distributed capabilities of Apache
>> > Hadoop® YARN via a simple and intuitive programming model similar to Java
>> > threads. Weave also has built-in capabilities required by many distributed
>> > applications, such as real-time application logs and metrics collection,
>> > application lifecycle management, and network service discovery.
>> >
>> > Background
>> > ==========
>> >
>> > Hadoop YARN is a generic cluster resource manager that supports any type of
>> > distributed application. However, YARN’s interfaces are too low level for
>> > rapid application development. It requires a great deal of boilerplate code
>> > even for a simple application, creating a high ramp up cost that can turn
>> > developers away.
>> >
>> > Weave is designed to improve this situation with a programming model that
>> > makes running distributed applications as easy as running Java threads.
>> > With the abstraction provided by Weave, applications can be executed in
>> > process threads during development and unit testing and then be deployed to
>> > a YARN cluster without any modifications.
>> >
>> > Weave also has built-in support for real-time application logs and metrics
>> > collection, delegation token renewal, application lifecycle management, and
>> > network service discovery. This greatly reduces the pain that developers
>> > face when developing, debugging, deploying and monitoring distributed
>> > applications.
>> >
>> > Weave is not a replacement for YARN, it’s a framework that operates on top
>> > of YARN.
>> >
>> > Rationale
>> > =========
>> >
>> > Developers who write YARN applications typically find themselves
>> > implementing the same (or similar) boilerplate code over and over again for
>> > every application. It makes sense to distill this common code into a
>> > reusable set of libraries that is perpetually maintained and improved by a
>> > diverse community of developers.
>> >
>> > Weave’s simple thread-like programming model will enable many Java
>> > programmers to develop distributed applications. We believe that this
>> > simplicity will attract developers who would otherwise be discouraged by
>> > complexity, and many new use cases will emerge for the usage of YARN.
>> >
>> > Incubating Weave as an Apache project makes sense because Weave is a
>> > framework built on top of YARN, and Weave uses Apache Zookeeper, HDFS,
>> > Kafka, and other Apache software (see the External Dependencies section).
>> >
>> > Current Status
>> > ==============
>> >
>> > Weave was initially developed at Continuuity. The Weave codebase is
>> > currently hosted in a public repository at github.com, which will seed the
>> > Apache git repository.
>> >
>> > Meritocracy
>> > -----------
>> > Our intent with this incubator proposal is to start building a diverse
>> > developer community around Weave following the Apache meritocracy model.
>> > Since Weave was initially developed in early 2013, we have had fast
>> > adoption and contributions within Continuuity. We are looking forward to
>> > new contributors. We wish to build a community based on Apache's
>> > meritocracy principles, working with those who contribute significantly to
>> > the project and welcoming them to be committers both during the incubation
>> > process and beyond.
>> >
>> > Community
>> > ---------
>> > Weave is currently being used internally at Continuuity and is at the core
>> > of our products. We hope to extend our contributor base significantly and
>> > we will invite all who are interested in simplifying the development of
>> > distributed applications to participate.
>> >
>> > Core Developers
>> > ---------------
>> > Weave is currently being developed by five engineers at Continuuity:
>> >  Terence Yim, Andreas Neumann, Gary Helmling, Poorna Chandra and Albert
>> > Shau.
>> > Terence Yim is an Apache committer for Helix, Andreas is an Apache
>> > committer and PMC member for Oozie, and Gary Helmling is an Apache
>> > committer and PMC member for HBase. Poorna Chandra and Albert Shau have
>> > made many contributions to Weave.
>> >
>> > Alignment
>> > ---------
>> > The ASF is the natural choice to host the Weave project as its goal of
>> > encouraging community-driven open source projects fits with our vision for
>> > Weave.
>> >
>> > Additionally, many other projects with which we are familiar and expect
>> > Weave to integrate with, such as ZooKeeper, YARN, HDFS, log4j, and others
>> > mentioned in the External Dependencies section are Apache projects, and
>> > Weave will benefit by close proximity to them.
>> >
>> > Known Risks
>> > ===========
>> >
>> > Orphaned Products
>> > -----------------
>> > There is very little risk of Weave being orphaned, as it is a key part of
>> > Continuuity’s products. The core Weave developers plan to continue to work
>> > on Weave, and Continuuity has funding in place to support their efforts
>> > going forward.
>> > Many other Big Data companies can benefit from Weave, and we have already
>> > received interest from various entities that would like to use and
>> > contribute to Weave.
>> >
>> > Inexperience with Open Source
>> > -----------------------------
>> > Several of the core developers have experience with open source
>> > development. Terence Yim, Andreas Neumann and Gary Helmling are currently
>> > Apache committers for Helix, Oozie and HBase respectively
>> > Homogeneous Developers
>> >
>> > The current core developers are all Continuuity employees. However, we
>> > intend to establish a developer community that includes independent and
>> > corporate contributors. We are encouraging new contributors via our mailing
>> > lists, public presentations, and personal contacts, and we will continue to
>> > do so. Various entities have already expressed interest in becoming
>> > involved with Weave.
>> >
>> > Reliance on Salaried Developers
>> > -------------------------------
>> > Currently, these developers are paid to work on Weave. Once the project has
>> > built a community, we expect to attract committers, developers and
>> > community other the current core developers. However, because Continuuity
>> > products use Weave internally, the reliance on salaried developers is
>> > unlikely to change, at least in the near term.
>> >
>> > Relationships with Other Apache Products
>> > ----------------------------------------
>> > Weave is deeply integrated with Apache projects. Weave uses Apache YARN as
>> > its underlying resource management and task scheduling system and Apache
>> > Zookeeper for coordination. In addition, Weave uses Apache HDFS and Apache
>> > Kafka. A number of other Apache projects are Weave dependencies and are
>> > listed in the External Dependencies section.
>> >
>> > An Excessive Fascination with the Apache Brand
>> > ----------------------------------------------
>> > While we respect the reputation of the Apache brand and have no doubt that
>> > it will attract contributors and users, our interest is primarily to give
>> > Weave a solid home as an open source project following an established
>> > development model. We have also given additional reasons in the Rationale
>> > and Alignment sections.
>> >
>> > Documentation
>> > =============
>> >
>> > https://github.com/continuuity/weave
>> >
>> > Initial Source
>> > ==============
>> >
>> > The Weave codebase is currently hosted at
>> > https://github.com/continuuity/weave.
>> >
>> > External Dependencies
>> > =====================
>> >
>> > The dependencies all have Apache-compatible licenses:
>> > • avro (Apache 2.0)
>> > • hadoop (Apache 2.0)
>> > • gson (Apache 2.0)
>> > • guava-libraries (Apache 2.0)
>> > • hbase (Apache 2.0)
>> > • hdfs (Apache 2.0)
>> > • kafka (Apache 2.0)
>> > • netty (Apache 2.0)
>> > • snappy-java (Apache 2.0)
>> > • yarn (Apache 2.0)
>> > • zookeeper (Apache 2.0)
>> > • asm (BSD)
>> > • junit (EPL v1.0)
>> > • logback (EPL v1.0 )
>> > • slf4j (MIT)
>> >
>> > Cryptography
>> > ============
>> >
>> > Weave will depend on secure Hadoop, which can optionally use Kerberos.
>> >
>> > Required Resources
>> > ==================
>> >
>> > Mailing Lists
>> > -------------
>> > - weave-private for private PMC discussions (with moderated subscriptions)
>> > - weave-dev for technical discussions among contributors
>> > - weave-commits for notification about commits
>> >
>> > Subversion Directory
>> > --------------------
>> > Git is the preferred source control system: git://git.apache.org/weave
>> >
>> > Issue Tracking
>> > --------------
>> > JIRA Weave (WEAVE)
>> >
>> > Other Resources
>> > ---------------
>> > The existing code already has unit tests, so we would like a Hudson
>> > instance to run them whenever a new patch is submitted. This can be added
>> > after project creation.
>> >
>> > Initial Committers
>> > ==================
>> >
>> > • Terence Yim
>> > • Andreas Neumann
>> > • Gary Helmling
>> > • Poorna Chandra
>> > • Albert Shau
>> >
>> > Affiliations
>> > ============
>> >
>> > • Terence Yim (Continuuity)
>> > • Andreas Neumann (Continuuity)
>> > • Gary Helmling (Continuuity)
>> > • Poorna Chandra (Continuuity)
>> > • Albert Shau (Continuuity)
>> >
>> > Sponsors
>> > ========
>> >
>> > Champion
>> > --------
>> > Vinod K <vinodkv at apache dot org> (Apache Member)
>> >
>> > Nominated Mentors
>> > -----------------
>> > • Arun C Murthy <acmurthy at apache dot org>
>> > • Tom White <tomwhite at apache dot org>
>> > • Patrick Hunt <phunt at apache dot org>
>> >
>> > Sponsoring Entity
>> > -----------------
>> > We are requesting that the Incubator sponsor this project.
>> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message