Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: error (athena.apache.org: local policy)
Message-ID: <521C4CC6.7030601@nanthrax.net>
Date: Tue, 27 Aug 2013 08:52:54 +0200
From: =?windows-1252?Q?Jean-Baptiste_Onofr=E9?= <jb@nanthrax.net>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:17.0) Gecko/20130803 Thunderbird/17.0.8
MIME-Version: 1.0
To: general@incubator.apache.org
Subject: Re: [PROPOSAL] Aurora for Incubation
References: 
 <CA+40=G0veax9MArbGDB5BjwGuGBj3ve0onrfHfMBrF1nkSw3Bw@mail.gmail.com>
In-Reply-To: 
 <CA+40=G0veax9MArbGDB5BjwGuGBj3ve0onrfHfMBrF1nkSw3Bw@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: quoted-printable

Hi guys,

the proposal is interesting.
Quick question: do you plan to focus only on Mesos, or create a kind of=20
more global and standalone job scheduler ?

Regards
JB

On 08/27/2013 12:27 AM, Dave Lester wrote:
> Hi All,
>
> We're pleased to share a draft ASF incubation proposal for Aurora, a
> service scheduler used to schedule jobs onto Apache Mesos that we've
> developed at Twitter. Aurora provides all of the primitives necessary t=
o
> quickly deploy and scale stateless and fault tolerant services in a
> datacenter. The complete proposal can be found:
> https://wiki.apache.org/incubator/AuroraProposal, and also pasted below=
.
>
> In particular, we'd love to add additional mentors to the project. Your
> feedback is appreciated.
>
> Dave
>
> =3D Abstract =3D
>
> Aurora is a service scheduler used to schedule jobs onto Apache Mesos.
>
> =3D Proposal =3D
>
> Aurora is a scheduler that provides all of the primitives necessary to
> quickly deploy and scale stateless and fault tolerant services in a
> datacenter.
>
> Aurora builds on top of Apache Mesos and provides common features that
> allow any site to run large scale production applications. While the
> project is currently used in production at Twitter, we wish to develop =
a
> community to increase contributions and see it thrive in the future.
>
> =3D Background =3D
>
> The initial development of Aurora was done at Twitter, and is planned t=
o be
> open sourced. This proposal is for Aurora to join the Apache Incubator.
>
> =3D Rationale =3D
>
> While the Apache Mesos core focuses on distributing individual tasks ac=
ross
> nodes in a cluster, typical services consist of dozens or hundreds of
> replicas of tasks. As a service scheduler, Aurora provides the abstract=
ion
> of a "job" to bundle and manage these tasks. Aurora provides many key
> functionalities centered around a job, including: definition, the conce=
pt
> of an instance and the serverset, deployment and scheduling, health
> checking, and introspection. It also allows cross-cutting concerns to b=
e
> handled like observability and log collection.
>
> =3D Current Status =3D
>
> =3D=3D Meritocracy =3D=3D
>
> By submitting this incubator proposal, we=92re expressing our intent to=
 build
> a diverse developer community around Aurora that will conduct itself
> according to The Apache Way and use meritocratic means of accepting
> contributions. Several members of the Aurora team overlap with Apache
> Mesos, which successfully graduated from the Incubator and has embraced=
 a
> meritocratic model of governance; we plan to follow a similar path forw=
ard
> with Aurora and believe that a synergy between both projects will make =
this
> even easier.
>
> =3D=3D Community =3D=3D
>
> Aurora is currently being used internally at Twitter. By open sourcing =
the
> project, we hope to extend our contributor base significantly and creat=
e a
> vibrant community around the project.
>
> =3D=3D Core Developers =3D=3D
>
> Aurora is currently being developed by a team of seven engineers at Twi=
tter.
>
> =3D=3D Alignment =3D=3D
>
> The ASF is a natural choice to host the Aurora project, given the goal =
of
> open sourcing the project and fostering a community to grow and support=
 the
> software. Additionally, Aurora integrates with Apache Mesos, and Apache
> ZooKeeper for service discovery.
>
> We believe that inclusion within Apache will build stronger ties betwee=
n
> these projects, and create further alignment between their goals and
> communities.
>
> =3D Known Risks =3D
>
> =3D=3D Orphaned Products =3D=3D
>
> The core developers plan to continue working full time on the project, =
and
> there is very little risk of Aurora being abandoned since it is running
> hundreds of services as part of Twitter=92s infrastructure. Additionall=
y,
> members of the Mesos community beyond Twitter have expressed interest i=
n an
> advanced scheduler like Aurora (see =93Interested Parties=94 section); =
we
> believe that need will drive some of the community involvement necessar=
y
> for the project to incubate successfully.
>
> =3D=3D Inexperience with Open Source =3D=3D
>
> Initial Aurora committers have varying levels of experience using and
> contributing to Open Source projects, however by working with our mento=
rs
> and the Apache community we believe we will be able to conduct ourselve=
s in
> accordance with Apache Incubator guidelines. The close relationship bet=
ween
> the Aurora team and Apache Mesos means there is an awareness of the
> incubation process and a willingness to embrace The Apache Way.
>
> =3D=3D Homogenous Developers =3D=3D
>
> The initial set of committers are from a single organization, however w=
e
> expect that once approved for incubation the project will attract
> contributors from more organizations. We have already had conversations
> with other companies who have expressed an interest in Aurora.
>
> =3D=3D Reliance on Salaried Developers =3D=3D
>
> Initial Aurora committers are salaried developers at Twitter, however
> shortly after open sourcing the code we plan to diversify the project=92=
s
> core committers and contributors.
>
> =3D=3D Relationships with Other Apache Products =3D=3D
>
> Initially, Aurora has been developed as a scheduler for Apache Mesos.
> Additionally, it relies on ZooKeeper for service discovery, allowing
> servers to register at a location and clients to subsequently discover =
the
> servers.
>
> =3D=3D An Excessive Fascination with the Apache Brand =3D=3D
>
> While we respect the reputation of the Apache brand and have no doubts =
that
> it will attract contributors and users, our interest is primarily to gi=
ve
> Aurora a solid home as an open source project following an established
> development model. We have also given reasons in the Rationale and
> Alignment sections.
>
> =3D Documentation =3D
>
> This proposal exists online as
> http://wiki.apache.org/incubator/AuroraProposal. Project documentation
> exists and will be released at the same time as an initial code push.
>
> =3D Initial Source =3D
>
> JIRA Aurora (AURORA)
>
> Source and Intellectual Property Submission Plan
> The current codebase is available upon request.
>
> =3D External Dependencies =3D
>
> All Aurora dependencies have Apache compatible licenses, except cron4j
> (LGPL). An alternative library, most likely Quartz (Apache Public Licen=
se
> 2.0) will be used prior to an initial code push.
>
> =3D Cryptography =3D
> Not applicable.
>
> =3D Required Resources =3D
>
> =3D=3D Mailing Lists =3D=3D
>
>   * aurora-private for private PMC discussions
>   * aurora-dev
>   * aurora-commits
>   * aurora-user
>   * aurora-issues
>
> =3D=3D Subversion Directory =3D=3D
>
> We prefer to use Git as our source control system: git://
> git.apache.org/aurora
>
> =3D=3D Issue Tracking =3D=3D
>
> JIRA Aurora (AURORA)
>
> =3D Initial Committers =3D
>
>   * William Farner (bill at twitter dot com)
>   * Brian Wickman (wickman at twitter dot com)
>   * Kevin Sweeney (ksweeney at twitter dot com)
>   * Sathya Hariesh (sathya at twitter dot com)
>   * Jonathan Boulle (jon at twitter dot com)
>   * Maxim Khutornenko (mkhutornenko at twitter dot com)
>   * Suman Karumuri (skarumuri at twitter dot com)
>   * Dave Lester (dlester at twitter dot com)
>
> =3D Affiliations =3D
>
> Twitter
>
> =3D Interested Parties =3D
>
>   * Airbnb
>   * Foursquare
>   * Mesosphere
>
> =3D Sponsors =3D
>
> =3D=3D Champion =3D=3D
>
> Ben Hindman
>
> =3D=3D Nominated Mentors =3D=3D
>
>   * Chris Mattmann
>   * Ben Hindman
>
> =3D=3D Sponsoring Entity =3D=3D
> Incubator PMC
>

--=20
Jean-Baptiste Onofr=E9
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org