Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of toelen@gmail.com designates
 209.85.219.216 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=fNfwWYx1LLsvfnJlJBQpbiUXUSAhpxI6zWLt738HnP4KtbsdIIvf5kVgFGBMXhcJbH
         vsv36EVuiGogUNis2d9olsKgMLRHu3w32Ky8JaVsP8VF9qOJ43EJFgdDved3eZkXwI0f
         VrEKhs2dHBmjvEaj8jUOMxePzBUxt5VFyGAWs=
MIME-Version: 1.0
In-Reply-To: <c93bd2771001281508q5edfe4e1v14e020549e606001@mail.gmail.com>
References: <7c962aed1001281238g2c49d660pf47d23fe90432886@mail.gmail.com>
	 <78568af11001281242i3419dda8ke7fc082d980df14c@mail.gmail.com>
	 <4aa34eb71001281427j69768786o755bc931ebf72245@mail.gmail.com>
	 <78568af11001281435l782926e5g4c180668ae0b7f8c@mail.gmail.com>
	 <c93bd2771001281508q5edfe4e1v14e020549e606001@mail.gmail.com>
Date: Fri, 29 Jan 2010 00:19:20 +0100
Message-ID: <a494a0bc1001281519g23a141f2j9bf6b870292cef39@mail.gmail.com>
Subject: Re: Discussion: Move contribs out of hbase?
From: Leen Toelen <toelen@gmail.com>
To: hbase-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001636c5b90aaa3fdc047e41bf91

--001636c5b90aaa3fdc047e41bf91
Content-Type: text/plain; charset=ISO-8859-1

Hi,

the continuous integration server could be configured to build and test all
contribs (no matter where they are located) against each hbase build.

Leen

On Fri, Jan 29, 2010 at 12:08 AM, Joydeep Sarma <jsensarma@gmail.com> wrote:

> fwiw - i agree with Dhruba's take. i can't imagine (for example)
> hadoop shipping without hadoop-streaming (which is contrib). and it's
> awesome that hadoop changes that break streaming are caught early and
> often.
>
> it makes sense this analog doesn't apply to all contrib projects -
> perhaps they shouldn't be in contrib to begin with? alternatively - to
> extend Dhruba's idea - one could break up contrib projects into those
> that are critical (like streaming would be to hadoop and i imagine the
> replication stuff would be to hbase) - and those that are non-critical
> (and not build these by default).
>
> the only reason to have non-critical contrib projects in the codebase
> seems to be to give young/new projects exposure. i am not sure whether
> this is a good enough reason - perhaps those with some historical
> perspective have better idea on this.
>
>
> On Thu, Jan 28, 2010 at 2:35 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> > Part of the problem with contrib is fixing the deeper problems during
> > major version changes (eg: new file format in 0.20) requires
> > specialized knowledge that only the contrib author has. Then they must
> > write a patch, submit it to a hbase committer, etc.  If the contrib
> > was in a non-ASF repo, the author could fix it much faster.
> >
> > That's my theory at least?
> >
> > On Thu, Jan 28, 2010 at 2:27 PM, Dhruba Borthakur <dhruba@gmail.com>
> wrote:
> >> Hi stack,
> >>
> >> I have seen this issue crop up in the HDFS  repository too.
> >>
> >> The issue that you have brought up is that code changes to core
> sometimes
> >> needs lots of changes to contrib to make it compile. This is a big
> problem,
> >> especially when all developers are not intricately familiar with each
> and
> >> every contrib module. However, the big plus is that when a HBase release
> is
> >> made, it has all the contrib modules in it, and it makes it easier for a
> >> user to start using any of those contrib modules right away. Can we get
> the
> >> best of both worlds if we do something like this:
> >>
> >>  Contribs are not first-class citizens. The build script can be setup to
> >> ignore build failures in contrib modules. The hope is that when a
> release is
> >> about to be cut, contrib owners will submit fixes to any build problems
> if
> >> they want their contrib to be shipped as part of the Hbase release. will
> >> this solve your problem?
> >>
> >> thanks,
> >> dhruba
> >>
> >>
> >>
> >> On Thu, Jan 28, 2010 at 12:42 PM, Ryan Rawson <ryanobjc@gmail.com>
> wrote:
> >>
> >>> +1
> >>>
> >>> I think when we are ready, we should publish the replication as a
> >>> contrib up on github. I really like github, its way better than any
> >>> other system I've seen (in part due to git of course).
> >>>
> >>> I do like the notion of officially released contribs that are included
> >>> at specific version numbers with the hbase release. Hbase core 0.21.0
> >>> with TH, ITH at version 0.1 or whatever.
> >>>
> >>> -ryan
> >>>
> >>> On Thu, Jan 28, 2010 at 12:38 PM, Stack <stack@duboce.net> wrote:
> >>> > I'd like to start a discussion on moving src/contrib out of hbase.
> >>> > Keep reading if you have an opinion.
> >>> >
> >>> > I'd like to suggest that we undo the notion of hbase contribs for the
> >>> > following reasons:
> >>> >
> >>> > + In my experience, they present a friction on changes to core as any
> >>> > significant core change tends to ripple down into the contribs
> whether
> >>> > its code or infrastructure/build changes.  Usually what happens then
> >>> > is a non-expert in the contrib code is making edits -- often radical
> >>> > -- to code they are not completely up on and are a little frustrated
> >>> > that they have to do it.  Bad.
> >>> > + A few of our contribs are maintained by non-committers.  This means
> >>> > it takes a committers time getting in updates.  The owner is at mercy
> >>> > of the committer making wanted changes.  The committer is consumed
> >>> > reviewing and making update.  This indirection hurts at both ends (We
> >>> > could discuss making contrib owners committers on their contrib only
> >>> > but that'd be a bit of bureaucratic nightmare and a burden on the
> >>> > hadoop pmc to vote on granting access to a subprojects, contrib.  Its
> >>> > tough enough getting hadoop pmc to vote on hbase committers.  There
> is
> >>> > no precedent in other project, to my knowledge).
> >>> > + Contribs and core evolve at different rates.  They should not be
> >>> > constrained by core release schedule (or the opposite, core should
> not
> >>> > be held up because fixes in contrib are wanting).
> >>> >
> >>> > I suggest that current contribs be moved out of hbase up to
> standalone
> >>> > github or google code projects (witness how hbql does it).  Previous
> >>> > to our move to Ivy (and possibly soon, Maven), asking contribs be
> >>> > standalone was a pain as they'd have to check in hbase jars and all
> of
> >>> > dependencies and then move these forward over time.  Now that we are
> >>> > Ivy-ized, contribs just need write a bit of ivy.xml and it'll take
> >>> > care of pulling dependencies.
> >>> >
> >>> > I'd imagine support for contribs would go on as it does now with
> >>> > queries up on hbase mailing list and help out on IRC.  We'd give
> >>> > contribs first-class billing up on home page.  Popular contribs might
> >>> > run their own mailing lists, etc...
> >>> >
> >>> > We should chat but some contribs should be pulled up into core.
> >>> > Thrift was core.  Talk was to move current thrift update out to
> >>> > contrib.  I think now it should just stay in core.  Stargate perhaps
> >>> > should come up into core?
> >>> >
> >>> > What do folks think?
> >>> > St.Ack
> >>> >
> >>> > P.S. I've fostered the contrib notion in the past.  I've since had a
> >>> > change of heart.  Please pardon my flip.
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Connect to me at http://www.facebook.com/dhruba
> >>
> >
>

--001636c5b90aaa3fdc047e41bf91--