Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of larry.mccay@gmail.com
 designates 74.125.82.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+RK=_DchwiB2EpJR+XBwDwzqeK4uz9ZHkS5MwJpDbjzPokjRQ@mail.gmail.com>
References: 
 <CANpmDOeayC5-7EeGVWgdc27Y++dNesiNz_e1znYHn98RW-tLaQ@mail.gmail.com>
	<CA+RK=_DchwiB2EpJR+XBwDwzqeK4uz9ZHkS5MwJpDbjzPokjRQ@mail.gmail.com>
Date: Fri, 8 Nov 2013 18:45:03 -0500
Message-ID: 
 <CACRbFyiFpubKNHJayx0xmBTuK0Sw1MWKcjMeVPDoGz0jo_uV7g@mail.gmail.com>
Subject: Re: [VOTE] Accept Twill for Incubation
From: larry mccay <larry.mccay@gmail.com>
To: general@incubator.apache.org
Content-Type: multipart/alternative; boundary=f46d043bdff8a01a3904eab2f84b

--f46d043bdff8a01a3904eab2f84b
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

+1 (non-binding)


On Fri, Nov 8, 2013 at 6:40 PM, Andrew Purtell <apurtell@apache.org> wrote:

> +1 (non-binding)
>
>
> On Thu, Nov 7, 2013 at 1:04 PM, Andreas Neumann <anew@apache.org> wrote:
>
> > The discussion about the Weave proposal has calmed. As the outcome of t=
he
> > discussion, we have chosen a new name for the project, Twill. I would
> like
> > to call a vote for Twill to become an incubated project.
> >
> > The proposal is pasted below, and also available at:
> > https://wiki.apache.org/incubator/TwillProposal
> >
> > Let's keep this vote open for three business days, closing the voting o=
n
> > Tuesday 11/12.
> >
> > [ ] +1 Accept Twill into the Incubator
> > [ ] +0 Don't care.
> > [ ] -1 Don't accept Twill because...
> >
> > -Andreas.
> >
> > =3D Abstract =3D
> >
> > Twill is an abstraction over Apache Hadoop=AE YARN that reduces the
> > complexity of developing distributed applications, allowing developers =
to
> > focus more on their business logic.
> >
> > =3D Proposal =3D
> >
> > Twill is a set of libraries that reduces the complexity of developing
> > distributed applications. It exposes the distributed capabilities of
> Apache
> > Hadoop=AE YARN via a simple and intuitive programming model similar to =
Java
> > threads. Twill also has built-in capabilities required by many
> distributed
> > applications, such as real-time application logs and metrics collection=
,
> > application lifecycle management, and network service discovery.
> >
> > =3D Background =3D
> >
> > Hadoop YARN is a generic cluster resource manager that supports any typ=
e
> of
> > distributed application. However, YARN=92s interfaces are too low level=
 for
> > rapid application development. It requires a great deal of boilerplate
> code
> > even for a simple application, creating a high ramp up cost that can tu=
rn
> > developers away.
> >
> > Twill is designed to improve this situation with a programming model th=
at
> > makes running distributed applications as easy as running Java threads.
> > With the abstraction provided by Twill, applications can be executed in
> > process threads during development and unit testing and then be deploye=
d
> to
> > a YARN cluster without any modifications.
> >
> > Twill also has built-in support for real-time application logs and
> metrics
> > collection, delegation token renewal, application lifecycle management,
> and
> > network service discovery. This greatly reduces the pain that developer=
s
> > face when developing, debugging, deploying and monitoring distributed
> > applications.
> >
> > Twill is not a replacement for YARN, it=92s a framework that operates o=
n
> top
> > of YARN.
> >
> > =3D Rationale =3D
> >
> > Developers who write YARN applications typically find themselves
> > implementing the same (or similar) boilerplate code over and over again
> > for every application. It makes sense to distill this common code into =
a
> > reusable set of libraries that is perpetually maintained and improved b=
y
> a
> > diverse community of developers.
> >
> > Twill=92s simple thread-like programming model will enable many Java
> > programmers to develop distributed applications. We believe that this
> > simplicity will attract developers who would otherwise be discouraged b=
y
> > complexity, and many new use cases will emerge for the usage of YARN.
> >
> > Incubating Twill as an Apache project makes sense because Twill is a
> > framework built on top of YARN, and Twill uses Apache Zookeeper, HDFS,
> > Kafka, and other Apache software (see the External Dependencies section=
).
> >
> > =3D Current Status =3D
> >
> > Twill was initially developed at Continuuity under the name of Weave. T=
he
> > Weave codebase is currently hosted in a public repository at github.com=
,
> > which will seed the Apache git repository after renaming to Twill.
> >
> > =3D=3D Meritocracy =3D=3D
> >
> > Our intent with this incubator proposal is to start building a diverse
> > developer community around Twill following the Apache meritocracy model=
.
> > Since Twill was initially developed in early 2013, we have had fast
> > adoption and contributions within Continuuity. We are looking forward t=
o
> > new contributors. We wish to build a community based on Apache's
> > meritocracy principles, working with those who contribute significantly
> to
> > the project and welcoming them to be committers both during the
> incubation
> > process and beyond.
> >
> > =3D=3D Community =3D=3D
> >
> > Twill is currently being used internally at Continuuity and is at the
> core
> > of our products. We hope to extend our contributor base significantly a=
nd
> > we will invite all who are interested in simplifying the development of
> > distributed applications to participate.
> >
> > =3D=3D Core Developers =3D=3D
> >
> > Twill is currently being developed by five engineers at Continuuity:
> > Terence Yim, Andreas Neumann, Gary Helmling, Poorna Chandra and Albert
> > Shau.
> > Terence Yim is an Apache committer for Helix, Andreas is an Apache
> > committer and PMC member for Oozie, and Gary Helmling is an Apache
> > committer and PMC member for HBase. Poorna Chandra and Albert Shau have
> > made many contributions to Twill.
> >
> > =3D=3D Alignment =3D=3D
> >
> > The ASF is the natural choice to host the Twill project as its goal of
> > encouraging community-driven open source projects fits with our vision
> for
> > Twill.
> >
> > Additionally, many other projects with which we are familiar and expect
> > Twill to integrate with, such as ZooKeeper, YARN, HDFS, log4j, and othe=
rs
> > mentioned in the External Dependencies section are Apache projects, and
> > Twill will benefit by close proximity to them.
> >
> > =3D Known Risks =3D
> >
> > =3D=3D Orphaned Products =3D=3D
> >
> > There is very little risk of Twill being orphaned, as it is a key part =
of
> > Continuuity=92s products. The core Twill developers plan to continue to
> work
> > on Twill, and Continuuity has funding in place to support their efforts
> > going forward.
> > Many other Big Data companies can benefit from Twill, and we have alrea=
dy
> > received interest from various entities that would like to use and
> > contribute to Twill.
> >
> > =3D=3D Inexperience with Open Source =3D=3D
> >
> > Several of the core developers have experience with open source
> > development. Terence Yim, Andreas Neumann and Gary Helmling are current=
ly
> > Apache committers for Helix, Oozie and HBase respectively
> > Homogeneous Developers
> >
> > The current core developers are all Continuuity employees. However, we
> > intend to establish a developer community that includes independent and
> > corporate contributors. We are encouraging new contributors via our
> mailing
> > lists, public presentations, and personal contacts, and we will continu=
e
> to
> > do so. Various entities have already expressed interest in becoming
> > involved with Twill.
> >
> > =3D=3D Reliance on Salaried Developers =3D=3D
> >
> > Currently, these developers are paid to work on Twill. Once the project
> has
> > built a community, we expect to attract committers, developers and
> > community
> > other than the current core developers. However, because Continuuity
> > products use Twill internally, the reliance on salaried developers is
> > unlikely to change, at least in the near term.
> >
> > =3D=3D Relationships with Other Apache Products =3D=3D
> >
> > Twill is deeply integrated with Apache projects. Twill uses Apache YARN
> as
> > its underlying resource management and task scheduling system and Apach=
e
> > Zookeeper for coordination. In addition, Twill uses Apache HDFS and
> Apache
> > Kafka. A number of other Apache projects are Twill dependencies and are
> > listed in the External Dependencies section.
> >
> > =3D=3D An Excessive Fascination with the Apache Brand =3D=3D
> >
> > While we respect the reputation of the Apache brand and have no doubt
> that
> > it will attract contributors and users, our interest is primarily to gi=
ve
> > Twill a solid home as an open source project following an established
> > development model. We have also given additional reasons in the Rationa=
le
> > and Alignment sections.
> >
> > =3D Documentation =3D
> >
> > The current documentation for Weave is at
> > https://github.com/continuuity/weave.
> > This will be adapted to the new name Twill.
> >
> > =3D Initial Source =3D
> >
> > The Weave codebase is currently hosted at
> > https://github.com/continuuity/weave. Weave will be renamed to Twill to
> > seed the Apache git repository.
> >
> > =3D External Dependencies =3D
> >
> > The dependencies all have Apache-compatible licenses:
> >  * avro (Apache 2.0)
> >  * hadoop (Apache 2.0)
> >  * gson (Apache 2.0)
> >  * guava-libraries (Apache 2.0)
> >  * hbase (Apache 2.0)
> >  * hdfs (Apache 2.0)
> >  * kafka (Apache 2.0)
> >  * netty (Apache 2.0)
> >  * snappy-java (Apache 2.0)
> >  * yarn (Apache 2.0)
> >  * zookeeper (Apache 2.0)
> >  * asm (BSD)
> >  * junit (EPL v1.0)
> >  * logback (EPL v1.0 )
> >  * slf4j (MIT)
> >
> > =3D Cryptography =3D
> >
> > Twill will depend on secure Hadoop, which can optionally use Kerberos.
> >
> > =3D Required Resources =3D
> >
> > =3D=3D Mailing Lists =3D=3D
> >
> >  * twill-private for private PMC discussions (with moderated
> subscriptions)
> >  * twill-dev for technical discussions among contributors
> >  * twill-commits for notification about commits
> >
> > =3D=3D Subversion Directory =3D=3D
> >
> > Git is the preferred source control system: git://git.apache.org/twill
> >
> > =3D=3D Issue Tracking =3D=3D
> >
> > JIRA Twill (TWILL)
> >
> > =3D=3D Other Resources =3D=3D
> >
> > The existing code already has unit tests, so we would like a Hudson
> > instance to run them whenever a new patch is submitted. This can be add=
ed
> > after project creation.
> >
> > =3D Initial Committers =3D
> >
> >  * Terence Yim
> >  * Andreas Neumann
> >  * Gary Helmling
> >  * Poorna Chandra
> >  * Albert Shau
> >
> > =3D Affiliations =3D
> >
> >  * Terence Yim (Continuuity)
> >  * Andreas Neumann (Continuuity)
> >  * Gary Helmling (Continuuity)
> >  * Poorna Chandra (Continuuity)
> >  * Albert Shau (Continuuity)
> >
> > =3D Sponsors =3D
> >
> > =3D=3D Champion =3D=3D
> >
> > Vinod K <vinodkv at apache dot org> (Apache Member)
> >
> > =3D=3D Nominated Mentors =3D=3D
> >
> >  * Arun C Murthy <acmurthy at apache dot org>
> >  * Tom White <tomwhite at apache dot org>
> >  * Patrick Hunt <phunt at apache dot org>
> >  * Andrei Savu <asavu at apache dot org>
> >
> > =3D=3D Sponsoring Entity =3D=3D
> >
> > We are requesting that the Incubator sponsor this project.
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

--f46d043bdff8a01a3904eab2f84b--