Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of ted.dunning@gmail.com
 designates 209.85.223.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CANyrgvcy7HaxV2BG4c3LyehcXy8UPmr+8K2bLsuFg5BJ4T5a1w@mail.gmail.com>
References: 
 <CA+ULb+vmgSao8v0BQxSecM7pAfrUWFyb42jAWxpDyf4Oka+Xeg@mail.gmail.com>
 <CANyrgvcy7HaxV2BG4c3LyehcXy8UPmr+8K2bLsuFg5BJ4T5a1w@mail.gmail.com>
From: Ted Dunning <ted.dunning@gmail.com>
Date: Wed, 5 Nov 2014 12:31:58 -0800
Message-ID: 
 <CAJwFCa3xXsKMmA5caeqfzGfPoMVdBA_A8UGZ1=0pdg99GCJ8Xg@mail.gmail.com>
Subject: Re: [VOTE] Accept HTrace into the Apache Incubator
To: "general@incubator.apache.org" <general@incubator.apache.org>,
 jfarrell@apache.org
Content-Type: multipart/alternative; boundary=001a113f94b281ab690507227aa1

--001a113f94b281ab690507227aa1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

+1 (binding)

On Wed, Nov 5, 2014 at 11:39 AM, Jake Farrell <jfarrell@apache.org> wrote:

> +1 (binding)
>
> -Jake
>
> On Wed, Nov 5, 2014 at 2:36 PM, Roman Shaposhnik <rvs@apache.org> wrote:
>
> > On Wed, Nov 5, 2014 at 11:16 AM, Roman Shaposhnik <rvs@apache.org>
> wrote:
> > > Following the discussion earlier in the thread:
> > >    http://s.apache.org/Dk7
> > >
> > > I would like to call a VOTE for accepting HTrace
> > > as a new incubator project.
> > >
> > > The proposal is available at:
> > >
> > > https://wiki.apache.org/incubator/HTraceProposal
> > >    (a full version of the proposal is attached)
> > >
> > > Vote is open until at least Sunday, 9th November 2014, 23:59:00 UTC
> > >
> > >  [ ] +1 accept Lens in the Incubator
> > >  [ ] =C2=B10
> > >  [ ] -1 because...
> >
> > Thanks,
> > Roman.
> >
> > =3D=3D Abstract =3D=3D
> > HTrace is a tracing framework intended for use with distributed
> > systems written in java.
> >
> > =3D=3D Proposal =3D=3D
> > HTrace is an aid for understanding system behavior and for reasoning
> > about performance
> > issues in distributed systems. HTrace is primarily a low impedance
> > library that a java
> > distributed system can incorporate to generate =E2=80=98breadcrumbs=E2=
=80=99 or
> > =E2=80=98traces=E2=80=99 along the path
> > of execution, even as it crosses processes and machines. HTrace also
> > includes various
> > tools and glue for collecting, processing and =E2=80=98visualizing=E2=
=80=99 captured
> > execution traces
> > for analysis ex post facto of where time was spent and what resources
> > were consumed.
> >
> > =3D=3D Background =3D=3D
> > Distributed systems are made up of multiple software components
> > running on multiple
> > computers connected by networks. Debugging or profiling operations run
> > over non-trivial
> > distributed systems -- figuring execution paths and what services,
> > machines, and
> > libraries participated in the processing of a request -- can be involve=
d.
> >
> > =3D=3D Rationale =3D=3D
> > Rather than have each distributed system build its own custom
> > =E2=80=98tracing=E2=80=99 libraries,
> > ideally all would use a single project that provides necessary
> > primitives and saves
> > each project building its own visualizations and processing tools anew.
> >
> > Google described =E2=80=9C...[a] large-scale distributed systems tracin=
g
> > infrastructure=E2=80=9D
> > in Dapper, a Large-Scale Distributed Systems Tracing Infrastructure. Th=
e
> > paper
> > tells a compelling story of what is possible when disparate systems
> > standardize
> > on a single tracing library and cooperate, =E2=80=98passing the baton=
=E2=80=99, filling
> out
> > trace context as executions cross systems.
> >
> > HTrace aims to provide a rough equivalent in open source of the describ=
ed
> > core
> > Dapper tools and library.  As it is adopted by more projects, there wil=
l
> > be a
> > =E2=80=98network effect=E2=80=99 as HTrace will provide a more comprehe=
nsive view of
> > activity
> > on the cluster.  For example, as HDFS gets HTrace support, we can conne=
ct
> > this
> > with the HTrace support in HBase to follow HBase requests as they enter
> > HDFS.
> >
> > Given the success of HTrace depends on its being integrated by many
> > projects,
> > HTrace should be perceived as unhampered, free of any commercial,
> > political,
> > or legal =E2=80=98taint=E2=80=99. Being an Apache project would help in=
 this regard.
> >
> > =3D=3D Initial Goals =3D=3D
> > HTrace is a small project of narrow scope but with a grand vision:
> >   * Move the HTrace source and repository to Apache, a vendor-neutral
> > location. Currently HTrace resides at a Cloudera-hosted repository.
> >   * Add past contributors as committers and institute Apache governance=
.
> >   * Evangelize and encourage HTrace diffusion. Initially we will
> > continue a focus on the Hadoop space since that is where most of the
> > initial contributors work and it is where HTrace has been initially
> > deployed.
> >   * Building out the standalone visualization tool that ships with
> HTrace.
> >   * Build more community and add more committers
> >
> > =3D=3D Current Status =3D=3D
> > Currently HTrace has a viable Java trace library that can be interpolat=
ed
> > to create =E2=80=98traces=E2=80=99.  The work that needs to be done on =
this library is
> > mostly
> > bug fixes, ease-of-use improvements, and performance tweaks.  In the
> > future,
> > we may add libraries for other languages besides Java.
> >
> > HTrace has means of dumping traces to the filesystem, Twitters=E2=80=99=
 Zipkin
> > (a tracing
> > sink and visualization system developed by Twitter
> > https://github.com/twitter/zipkin),
> > or Apache HBase.  Executions can be viewed either in Zipkin or in pygra=
ph
> > (https://code.google.com/p/python-graph/).
> >
> > Since the initial sprint in the summer of 2012 which saw HTrace patches
> > proposed
> > for Apache HDFS and committed to Apache HBase, development has been
> > sporadic;
> > mostly a single developer or two adding a feature or bug fixing. HTrace
> is
> > currently undergoing a new =E2=80=9Cspurt=E2=80=9D of development with =
the effort to get
> > HTrace
> > added to Apache HDFS revived and a new standalone viewing facility bein=
g
> > added
> > in to HTrace itself.
> >
> > HTrace has been integrated by Apache Phoenix.
> >
> >
> > =3D=3D=3D Meritocracy =3D=3D=3D
> > HTrace, up to this, has been run by Apache committers and PMC members.
> > We want to
> > build out a diverse developer and user community and run the HTrace
> > project in
> > the Apache way.  Users and new contributors will be treated with respec=
t
> > and
> > welcomed; they will earn merit in the project by tendering quality
> patches
> > and support that move the project forward.  Those with a proven support
> and
> > quality patch track record will be encouraged to become committers.
> >
> > =3D=3D=3D Community =3D=3D=3D
> > There are just a few developers involved at the moment. If our project
> > is accepted
> > by incubator, building community would be a primary initial goal.
> >
> > =3D=3D=3D Core Developers =3D=3D=3D
> >
> > Core developers include Apache members and members of the Hadoop and
> > HBase PMCs.
> > Of those listed, all have contributed to HTrace. Half are from Cloudera=
.
> > The remainder are Hortonworks, NTTData, Google, and Facebook employees.
> >
> > =3D=3D=3D Alignment =3D=3D=3D
> > HTrace has been integrated into Apache HBase and Apache Phoenix.
> > Integration
> > into Apache HDFS is currently being worked on. Approaching the Apache
> YARN
> > project would be a likely next integration.
> >
> >
> > =3D=3D Known Risks =3D=3D
> > As noted above, development has been sporadic up to this.  It may
> continue
> > so.
> >
> > For HTrace to tell a compelling story, it needs to be taken up by
> > significant
> > projects that make up a traced distributed system.  For example, say YA=
RN
> > and
> > HBase take on HTrace but HDFS does not, then the HDFS portions of an
> > end-to-end
> > operation will render opaque compromising our being able to tell a good
> > story
> > around an execution. Because the picture painted has gaps, HTrace may b=
e
> > left
> > aside as ineffective.
> >
> > =3D=3D=3D Orphaned products =3D=3D=3D
> > The proposers have a vested interest in making HTrace succeed, driving
> its
> > development and its insertion into projects we all work on. Its
> dispersion
> > will shine light on difficult to understand interactions amongst the
> > various
> > systems we all work on. A working, integrated HTrace will add a useful
> > debugging mechanism to the Apache projects we all work on.
> >
> >
> > =3D=3D=3D Inexperience with Open Source =3D=3D=3D
> > The majority of the proposers here have day jobs that has them working
> near
> > full-time on (Apache) open source projects. A few of us have helped car=
ry
> > other projects through incubator.  HTrace to date has been developed as
> > an open source project.
> >
> > =3D=3D=3D Homogenous Developers =3D=3D=3D
> > The initial group of committers is small but already we have a healthy
> > diversity of participating companies.  We are bay-area challenged but
> > a Japanese contributor makes for a good counter balance.
> >
> > =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
> > Most of the contributors are paid to work in the Hadoop ecosystem.
> > While we might wander from our current employers, we probably won=E2=80=
=99t
> > go far from the Hadoop tree.  Whoever the Hadoop employer, it is
> > plain a successful HTrace project is in everyone=E2=80=99s interest.
> > At least one of the developers has already changed employers but
> > his interest in seeing HTrace succeed prevails.
> >
> > =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D
> > For HTrace to succeed, it is critical we build good relations with
> > other distributed systems projects.  We intend to initially build
> > on relations we already have in place, mostly in the Hadoop space.
> >
> > The HTrace project has been incorporated by Apache HBase and
> > Apache Phoenix. It is currently being actively integrated into
> > Apache HDFS.
> >
> > We do not know of any equivalent or near-equivalent project
> > in the Apache space.
> >
> > The Dapper paper notes precedent, in particular, the Berkeley
> > Rad Lab X-Trace project.
> >
> > =3D=3D=3D=3D How HTrace relates to Zipkin =3D=3D=3D=3D
> > Zipkin is an Apache Licensed project from Twitter. It is a complete
> > tracing tool with trace collectors, trace viewers and tools to help
> > you generate traces. It is written in Scala.  If your project is
> > not Scala or if it is Java and you cannot afford a Scala dependency,
> > at a minimum, you need an alternate means of generating traces.
> > HTrace provides this facility for Java as well as bridging tools
> > to feed traces to Zipkin for query and display.
> >
> > The projects complement each other.
> >
> > =3D=3D=3D A Excessive Fascination with the Apache Brand =3D=3D=3D
> > While we intend to leverage the Apache =E2=80=98branding=E2=80=99 when =
talking to other
> > projects as testament of our project=E2=80=99s =E2=80=98neutrality=E2=
=80=99, we have no plans
> > for making use of Apache brand in press releases nor posting billboards
> > advertising acceptance of HTrace into Apache Incubator.
> >
> >
> > =3D=3D Documentation =3D=3D
> > See [[http://htrace.org|htrace.org]] for the current state of the HTrac=
e
> > project and documentation.
> >
> > How to enable tracing in
> > [[http://hbase.apache.org/book/tracing.html|HBase using HTrace]]
> > Elliott Clark on
> > [[
> http://files.meetup.com/1350427/HBase%20Meetup%20-%20Zipkin.pptx|tracing
> > in HBase]]
> >
> > =3D=3D Initial Source =3D=3D
> > Jonathan Leavitt and Todd Lipcon built the first versions of HTrace in
> the
> > summer of 2012.  Jonathan was Todd=E2=80=99s summer intern at Cloudera.
> >
> >
> > =3D=3D Source and Intellectual Property Submission Plan =3D=3D
> > We know of no legal encumberments in the way of transfer of source to
> > Apache.
> >
> > =3D=3D External Dependencies =3D=3D
> > HTrace includes third party libs. These include guava, jetty, junit,
> > protobuf,
> > hbase, and thrift.  All dependencies are Apache licensed or licenses th=
at
> > are
> > palatable: e.g. junit is EPL (Eclipse Public License v1.0) and
> > ProtoBufs are BSD licensed.
> >
> > Cryptography
> > N/A
> >
> > =3D=3D Required Resources =3D=3D
> >
> > =3D=3D=3D Mailing lists =3D=3D=3D
> >   * private@htrace.incubator.apache.org (moderated subscriptions)
> >   * commits@htrace.incubator.apache.org
> >   * dev@htrace.incubator.apache.org
> >   * issues@htrace.incubator.apache.org
> >   * user@htrace.incubator.apache.org
> >
> > =3D=3D=3D Git Repository =3D=3D=3D
> > https://git-wip-us.apache.org/repos/asf/incubator-htrace.git
> >
> > =3D=3D=3D Issue Tracking =3D=3D=3D
> > JIRA HTrace (HTRACE)
> >
> > =3D=3D=3D Other Resources =3D=3D=3D
> > Means of setting up regular builds for htrace on builds.apache.org
> >
> > =3D=3D Initial Committers =3D=3D
> >   * Colin McCabe (cmccabe@apache.org)
> >   * Elliott Clark (eclark@apache.org)
> >   * Jonathan Leavitt (jon.s.leavitt@gmail.com) -- CLA being submitted
> >   * Masatake Iwasaki (iwasakims@gmail.com) -- CLA being submitted
> >   * Michael Stack (stack@apache.org)
> >   * Nick Dimiduk (ndimiduk@apache.org)
> >   * Todd Lipcon (todd@apache.org)
> >
> >
> > =3D=3D Affiliations =3D=3D
> >   * Colin McCabe - Cloudera
> >   * Elliott Clark - Facebook
> >   * Jonathan Leavitt - Google
> >   * Masatake Iwasaki - NTTData
> >   * Michael Stack - Cloudera
> >   * Nick Dimiduk - Hortonworks
> >   * Todd Lipcon - Cloudera
> >
> > =3D=3D Sponsors =3D=3D
> >
> > =3D=3D=3D Champion =3D=3D=3D
> > Roman Shaposhnik
> >
> > =3D=3D=3D Nominated Mentors =3D=3D=3D
> >   * Michael Stack - Apache Member
> >   * Todd Lipcon - Apache Member
> >   * Jake Farrell - Apache Member
> >   * Billie Rinaldi - Apache Member
> >   * Andrew Purtell - Apache Member
> >   * Lewis John McGibbney - Apache Member
> >
> >
> > We will be soliciting more mentors as part of the proposal process.
> >
> > =3D=3D=3D Sponsoring Entity =3D=3D=3D
> > We would like to propose Apache incubator to sponsor this project.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
> >
>

--001a113f94b281ab690507227aa1--