Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (nike.apache.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
Message-ID: <527E36B1.5010004@nanthrax.net>
Date: Sat, 09 Nov 2013 14:20:49 +0100
From: =?windows-1252?Q?Jean-Baptiste_Onofr=E9?= <jb@nanthrax.net>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: general@incubator.apache.org
Subject: Re: [VOTE] Accept Twill for Incubation
References: 
 <CANpmDOeayC5-7EeGVWgdc27Y++dNesiNz_e1znYHn98RW-tLaQ@mail.gmail.com>
 <CA++geaBJVOYpBO1_FRWz6tHjRcY+CT77yrGmyS4NmBX4WOFRNg@mail.gmail.com>
In-Reply-To: 
 <CA++geaBJVOYpBO1_FRWz6tHjRcY+CT77yrGmyS4NmBX4WOFRNg@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: quoted-printable

+1 (binding)

Regards
JB

On 11/08/2013 06:49 AM, Ashish wrote:
> +1 (non-binding)
>
>
> On Fri, Nov 8, 2013 at 2:34 AM, Andreas Neumann <anew@apache.org> wrote=
:
>
>> The discussion about the Weave proposal has calmed. As the outcome of =
the
>> discussion, we have chosen a new name for the project, Twill. I would =
like
>> to call a vote for Twill to become an incubated project.
>>
>> The proposal is pasted below, and also available at:
>> https://wiki.apache.org/incubator/TwillProposal
>>
>> Let's keep this vote open for three business days, closing the voting =
on
>> Tuesday 11/12.
>>
>> [ ] +1 Accept Twill into the Incubator
>> [ ] +0 Don't care.
>> [ ] -1 Don't accept Twill because...
>>
>> -Andreas.
>>
>> =3D Abstract =3D
>>
>> Twill is an abstraction over Apache Hadoop=AE YARN that reduces the
>> complexity of developing distributed applications, allowing developers=
 to
>> focus more on their business logic.
>>
>> =3D Proposal =3D
>>
>> Twill is a set of libraries that reduces the complexity of developing
>> distributed applications. It exposes the distributed capabilities of A=
pache
>> Hadoop=AE YARN via a simple and intuitive programming model similar to=
 Java
>> threads. Twill also has built-in capabilities required by many distrib=
uted
>> applications, such as real-time application logs and metrics collectio=
n,
>> application lifecycle management, and network service discovery.
>>
>> =3D Background =3D
>>
>> Hadoop YARN is a generic cluster resource manager that supports any ty=
pe of
>> distributed application. However, YARN=92s interfaces are too low leve=
l for
>> rapid application development. It requires a great deal of boilerplate=
 code
>> even for a simple application, creating a high ramp up cost that can t=
urn
>> developers away.
>>
>> Twill is designed to improve this situation with a programming model t=
hat
>> makes running distributed applications as easy as running Java threads=
.
>> With the abstraction provided by Twill, applications can be executed i=
n
>> process threads during development and unit testing and then be deploy=
ed to
>> a YARN cluster without any modifications.
>>
>> Twill also has built-in support for real-time application logs and met=
rics
>> collection, delegation token renewal, application lifecycle management=
, and
>> network service discovery. This greatly reduces the pain that develope=
rs
>> face when developing, debugging, deploying and monitoring distributed
>> applications.
>>
>> Twill is not a replacement for YARN, it=92s a framework that operates =
on top
>> of YARN.
>>
>> =3D Rationale =3D
>>
>> Developers who write YARN applications typically find themselves
>> implementing the same (or similar) boilerplate code over and over agai=
n
>> for every application. It makes sense to distill this common code into=
 a
>> reusable set of libraries that is perpetually maintained and improved =
by a
>> diverse community of developers.
>>
>> Twill=92s simple thread-like programming model will enable many Java
>> programmers to develop distributed applications. We believe that this
>> simplicity will attract developers who would otherwise be discouraged =
by
>> complexity, and many new use cases will emerge for the usage of YARN.
>>
>> Incubating Twill as an Apache project makes sense because Twill is a
>> framework built on top of YARN, and Twill uses Apache Zookeeper, HDFS,
>> Kafka, and other Apache software (see the External Dependencies sectio=
n).
>>
>> =3D Current Status =3D
>>
>> Twill was initially developed at Continuuity under the name of Weave. =
The
>> Weave codebase is currently hosted in a public repository at github.co=
m,
>> which will seed the Apache git repository after renaming to Twill.
>>
>> =3D=3D Meritocracy =3D=3D
>>
>> Our intent with this incubator proposal is to start building a diverse
>> developer community around Twill following the Apache meritocracy mode=
l.
>> Since Twill was initially developed in early 2013, we have had fast
>> adoption and contributions within Continuuity. We are looking forward =
to
>> new contributors. We wish to build a community based on Apache's
>> meritocracy principles, working with those who contribute significantl=
y to
>> the project and welcoming them to be committers both during the incuba=
tion
>> process and beyond.
>>
>> =3D=3D Community =3D=3D
>>
>> Twill is currently being used internally at Continuuity and is at the =
core
>> of our products. We hope to extend our contributor base significantly =
and
>> we will invite all who are interested in simplifying the development o=
f
>> distributed applications to participate.
>>
>> =3D=3D Core Developers =3D=3D
>>
>> Twill is currently being developed by five engineers at Continuuity:
>> Terence Yim, Andreas Neumann, Gary Helmling, Poorna Chandra and Albert
>> Shau.
>> Terence Yim is an Apache committer for Helix, Andreas is an Apache
>> committer and PMC member for Oozie, and Gary Helmling is an Apache
>> committer and PMC member for HBase. Poorna Chandra and Albert Shau hav=
e
>> made many contributions to Twill.
>>
>> =3D=3D Alignment =3D=3D
>>
>> The ASF is the natural choice to host the Twill project as its goal of
>> encouraging community-driven open source projects fits with our vision=
 for
>> Twill.
>>
>> Additionally, many other projects with which we are familiar and expec=
t
>> Twill to integrate with, such as ZooKeeper, YARN, HDFS, log4j, and oth=
ers
>> mentioned in the External Dependencies section are Apache projects, an=
d
>> Twill will benefit by close proximity to them.
>>
>> =3D Known Risks =3D
>>
>> =3D=3D Orphaned Products =3D=3D
>>
>> There is very little risk of Twill being orphaned, as it is a key part=
 of
>> Continuuity=92s products. The core Twill developers plan to continue t=
o work
>> on Twill, and Continuuity has funding in place to support their effort=
s
>> going forward.
>> Many other Big Data companies can benefit from Twill, and we have alre=
ady
>> received interest from various entities that would like to use and
>> contribute to Twill.
>>
>> =3D=3D Inexperience with Open Source =3D=3D
>>
>> Several of the core developers have experience with open source
>> development. Terence Yim, Andreas Neumann and Gary Helmling are curren=
tly
>> Apache committers for Helix, Oozie and HBase respectively
>> Homogeneous Developers
>>
>> The current core developers are all Continuuity employees. However, we
>> intend to establish a developer community that includes independent an=
d
>> corporate contributors. We are encouraging new contributors via our ma=
iling
>> lists, public presentations, and personal contacts, and we will contin=
ue to
>> do so. Various entities have already expressed interest in becoming
>> involved with Twill.
>>
>> =3D=3D Reliance on Salaried Developers =3D=3D
>>
>> Currently, these developers are paid to work on Twill. Once the projec=
t has
>> built a community, we expect to attract committers, developers and
>> community
>> other than the current core developers. However, because Continuuity
>> products use Twill internally, the reliance on salaried developers is
>> unlikely to change, at least in the near term.
>>
>> =3D=3D Relationships with Other Apache Products =3D=3D
>>
>> Twill is deeply integrated with Apache projects. Twill uses Apache YAR=
N as
>> its underlying resource management and task scheduling system and Apac=
he
>> Zookeeper for coordination. In addition, Twill uses Apache HDFS and Ap=
ache
>> Kafka. A number of other Apache projects are Twill dependencies and ar=
e
>> listed in the External Dependencies section.
>>
>> =3D=3D An Excessive Fascination with the Apache Brand =3D=3D
>>
>> While we respect the reputation of the Apache brand and have no doubt =
that
>> it will attract contributors and users, our interest is primarily to g=
ive
>> Twill a solid home as an open source project following an established
>> development model. We have also given additional reasons in the Ration=
ale
>> and Alignment sections.
>>
>> =3D Documentation =3D
>>
>> The current documentation for Weave is at
>> https://github.com/continuuity/weave.
>> This will be adapted to the new name Twill.
>>
>> =3D Initial Source =3D
>>
>> The Weave codebase is currently hosted at
>> https://github.com/continuuity/weave. Weave will be renamed to Twill t=
o
>> seed the Apache git repository.
>>
>> =3D External Dependencies =3D
>>
>> The dependencies all have Apache-compatible licenses:
>>   * avro (Apache 2.0)
>>   * hadoop (Apache 2.0)
>>   * gson (Apache 2.0)
>>   * guava-libraries (Apache 2.0)
>>   * hbase (Apache 2.0)
>>   * hdfs (Apache 2.0)
>>   * kafka (Apache 2.0)
>>   * netty (Apache 2.0)
>>   * snappy-java (Apache 2.0)
>>   * yarn (Apache 2.0)
>>   * zookeeper (Apache 2.0)
>>   * asm (BSD)
>>   * junit (EPL v1.0)
>>   * logback (EPL v1.0 )
>>   * slf4j (MIT)
>>
>> =3D Cryptography =3D
>>
>> Twill will depend on secure Hadoop, which can optionally use Kerberos.
>>
>> =3D Required Resources =3D
>>
>> =3D=3D Mailing Lists =3D=3D
>>
>>   * twill-private for private PMC discussions (with moderated subscrip=
tions)
>>   * twill-dev for technical discussions among contributors
>>   * twill-commits for notification about commits
>>
>> =3D=3D Subversion Directory =3D=3D
>>
>> Git is the preferred source control system: git://git.apache.org/twill
>>
>> =3D=3D Issue Tracking =3D=3D
>>
>> JIRA Twill (TWILL)
>>
>> =3D=3D Other Resources =3D=3D
>>
>> The existing code already has unit tests, so we would like a Hudson
>> instance to run them whenever a new patch is submitted. This can be ad=
ded
>> after project creation.
>>
>> =3D Initial Committers =3D
>>
>>   * Terence Yim
>>   * Andreas Neumann
>>   * Gary Helmling
>>   * Poorna Chandra
>>   * Albert Shau
>>
>> =3D Affiliations =3D
>>
>>   * Terence Yim (Continuuity)
>>   * Andreas Neumann (Continuuity)
>>   * Gary Helmling (Continuuity)
>>   * Poorna Chandra (Continuuity)
>>   * Albert Shau (Continuuity)
>>
>> =3D Sponsors =3D
>>
>> =3D=3D Champion =3D=3D
>>
>> Vinod K <vinodkv at apache dot org> (Apache Member)
>>
>> =3D=3D Nominated Mentors =3D=3D
>>
>>   * Arun C Murthy <acmurthy at apache dot org>
>>   * Tom White <tomwhite at apache dot org>
>>   * Patrick Hunt <phunt at apache dot org>
>>   * Andrei Savu <asavu at apache dot org>
>>
>> =3D=3D Sponsoring Entity =3D=3D
>>
>> We are requesting that the Incubator sponsor this project.
>>
>
>
>

--=20
Jean-Baptiste Onofr=E9
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org