Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of mslee@facebook.com designates
 204.15.23.140 as permitted sender)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: [PROPOSAL] Thrift
Date: Mon, 28 Jan 2008 21:23:37 -0800
Message-ID: 
 <4B6D3FD0996CC840AEE0BF81F7ED510602CA532E@sf2pmxb02.TheFacebook.com>
in-reply-to: 
 <4B6D3FD0996CC840AEE0BF81F7ED510602CA4BAC@sf2pmxb02.TheFacebook.com>
Thread-Topic: [PROPOSAL] Thrift
Thread-Index: AcheLdg04EbIbUc2T5C2veSKNsZJoAECMSig
From: "Mark Slee" <mslee@facebook.com>
To: <general@incubator.apache.org>
Cc: <thrift@publists.facebook.com>

Hi everyone,

Unless my mail filters are playing tricks on me, haven't seen any
feedback on the incubator list since sending this proposal out last
week. Hopefully that's a good thing, meaning no one's opposed. =3D)

Anyone have feedback? If no objections are voiced, when would it be
appropriate for us to move forward with a VOTE thread on this?

Cheers,
Mark

-----Original Message-----
From: Mark Slee [mailto:mslee@facebook.com]=20
Sent: Wednesday, January 23, 2008 6:07 PM
To: general@incubator.apache.org
Cc: thrift@publists.facebook.com
Subject: [PROPOSAL] Thrift

Hi all,
=20
We've just posted the Apache Incubator proposal for Thrift onto the
Wiki:
http://wiki.apache.org/incubator/ThriftProposal
=20
For those who prefer reading Wiki markup directly, it is copied below.
=20
Cheers,
Mark
=20
=3D Thrift Proposal =3D

=3D=3D Abstract =3D=3D
Thrift is a framework for efficient cross-language data serialization,
RPC, and server programming.

=3D=3D Proposal =3D=3D
Thrift is a software library and set of code-generation tools designed
to expedite development and implementation of efficient and scalable
backend services. Its primary goal is to enable efficient and reliable
communication across programming languages by abstracting the portions
of each language that tend to require the most customization into a
common library that is implemented in each language. Specifically,
Thrift allows developers to define datatypes and service interfaces in a
single language-neutral file and generate all the necessary code to
build RPC clients and servers.

=3D=3D Background =3D=3D
Thrift was initially developed at Facebook starting in 2006 to power RPC
and data logging for a number of backend services for the site, such as
Search and News Feed. The package was designed for open source, and was
released in early 2007. Since then, a number of other developers have
submitted patches to the project and become de facto owners of major
parts. Support for many languages has been developed entirely outside of
Facebook.

=3D=3D Rationale =3D=3D
The need for high performance, reliable communication across different
programming languages seems to be growing more and more common in modern
programming, particularly when writing software for the web.
Historically, this problem has forced developers to standardize on one
language/framework or adopt heavier-weight systems, such as CORBA or
SOAP. These systems tend to make tradeoffs that aren't always ideal for
the use case. SOAP, for example, may be ideal for calling across
disparate web services, but is unnecessarily verbose for service calls
on an intranet.

Most of these systems also require developers to learn the particulars
of their type systems, especially when dealing with containers or
objects. One of the primary goals of Thrift is to allow developers to
program across languages while still using the standard idioms and style
in each language. Custom type systems also makes code reuse more
difficult. Thrift allows developers to avoid creating unnecessary
wrapper interfaces by operating directly on native types.

=3D=3D Initial Goals =3D=3D
Though the bulk of Thrift's initial development is complete, there are
still some large areas for future development. Some areas we hope to
focus on in Apache:

 * Better log storage/replay
 * Meta-data serialization
 * Higher-performance serialization, standard C extension model across
Python/PHP/etc.
 * Extending the abstraction to a multi-client that can fan-out across
multiple servers

=3D=3D Current Status =3D=3D
=3D=3D=3D Meritocracy =3D=3D=3D
Though initial development was done at Facebook, Thrift was intended to
be released as an open source project from its inception. Since release,
many developers have adopted the framework and submitted significant
patches. Large portions of the codebase are now managed by those most
familiar with and responsible for them. Any potentially controversial
change is discussed on the public mailing list
(http://lists.pub.facebook.com/mailman/listinfo/thrift/) and good
suggestions are frequently implemented.

=3D=3D=3D Community =3D=3D=3D
Thrift is currently in use across a number of organizations, and we
expect this to grow as Thrift becomes a relevant and useful tool for
building more open source projects.

=3D=3D=3D Core Developers =3D=3D=3D
Thrift currently has developers across many organizations (e.g.
Facebook, Powerset, ReCaptcha, AmieStreet), many of whom are
contributors to other open source projects.

=3D=3D=3D Alignment =3D=3D=3D
Thrift is designed to integrate cleanly with other projects. We think
this is a particularly good fit for Apache due to integration potential
with other projects, specifically Hadoop/Hbase.

=3D=3D Known Risks =3D=3D
=3D=3D=3D Orphaned Products =3D=3D=3D
Thrift is already deployed into production at multiple large websites
that are frequently iterating on the featureset. There's no realistic
chance of it becoming orphaned.

=3D=3D=3D Inexperience with Open Source =3D=3D=3D
The project has already been open source for nearly a year and has
attracted many developers already. Part of the reason to join Apache is
to make the project work even better as open source by removing some
obstacles, such as Facebook hosting the SVN, and putting the resources
all in a truly open space, being able to have more committers, etc. Most
of the core developers have a history of working with open source tools.

=3D=3D=3D Homogenous Developers =3D=3D=3D
The current set of developers work across a variety of organizations.
Naturally, most are websites with significant backend structure (and
hence a need for Thrift), but the problems they are solving are diverse,
and many don't work in the same programming languages.

=3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
Thrift is a "means to an end." None of the developers (to my knowledge)
working on Thrift are salaried specifically to work on Thrift. Rather,
Thrift is useful in building other projects, which may or may not be for
salary. Realistically, it is likely that a decent portion of work on
Thrift will be done by someone at a company, but not specifically tasked
with working on Thrift. So long as the tool is relevant and useful, this
should result in developers contributing time both at work and
personally.

=3D=3D=3D Relationships with Other Apache Projects =3D=3D=3D Thrift has =
already been
introduced into the Hbase project. (See
https://issues.apache.org/jira/browse/HADOOP-2389) Since Thrift is a
development tool, it is designed and well-suited for use in other
projects. As a start, we definitely plan to continue integration work
with Hbase.

=3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D =
Thrift has
already attracted a stable base of developers. The reasons for joining
Apache are not to advertise the project, but rather to demonstrate the
commitment to open source by divorcing the trunk from any one
corporation and pursuing further integration with other Apache projects.

=3D=3D Documentation =3D=3D
Existing page:
http://developers.facebook.com/thrift/

Mailing list (with archives):
http://lists.pub.facebook.com/mailman/listinfo/thrift/

=3D=3D Initial Source =3D=3D
Currently hosted by Facebook:
http://svn.facebook.com/svnroot/thrift/

=3D=3D Source and Intellectual Property Submission Plan =3D=3D All code
currently hosted in the Facebook public SVN folder will be contributed.

=3D=3D External Dependencies =3D=3D
All dependencies (libevent, Boost) have Apache compatible licenses.

=3D=3D Cryptography =3D=3D
None

=3D=3D Required Resources =3D=3D
=3D=3D=3D Mailing lists =3D=3D=3D
 * thrift-dev
 * thrift-private

=3D=3D=3D Subversion Directory =3D=3D=3D
https://svn.apache.org/repos/asf/incubator/thrift

We'd also be interested in using git to store the repo. Does apache have
infrastucture set up to support that? It'd make it easier for
non-committer developers to work on patches, checkpoint commits, etc.

=3D=3D=3D Issue Tracking =3D=3D=3D
Bugzilla Thrift

=3D=3D Initial Committers =3D=3D
 * Mark Slee (mcslee at facebook dot com)
 * Aditya Agarwal (aditya at facebook dot com)
 * Marc Kwiatkowski (marc at facebook dot com)
 * David Reiss (david at facebook dot com)
 * James Wang (jwang at facebook dot com)
 * Chris Piro (cpiro at facebook dot com)
 * Ben Maurer (bmaurer at andrew dot cmu dot edu)
 * Kevin Clark (kevin at powerset dot com)
 * Jake Luciani (jakers at gmail dot com)

=3D=3D Affiliations =3D=3D
 * People with Facebook email addresses - Facebook
 * Ben Maurer - ReCaptcha
 * Kevin Clark - Powerset

=3D=3D Sponsors =3D=3D
=3D=3D=3D Champion =3D=3D=3D
 * Doug Cutting

=3D=3D=3D Mentors =3D=3D=3D
 * Paul Querna
 * Upayavira

=3D=3D=3D Sponsoring Entity =3D=3D=3D
 * Incubator


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org