Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of jsensarma@gmail.com
 designates 209.85.217.215 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=xBDisIS3hjEjiQkNZ/euC70LH6s2/mSX9RoGT1UKGwsgGoeVdIiAE+nfutbZd5MhWG
         QdgcwR3tKp8x+vLmbzDCq6U3tfOw1HkEs+N8os2sYUvzktUsSu6EjwBeLcBsqbdU6vsq
         CzTAGqTeHbDGzkCykZ6eYXUKz7nuxMSsnESy0=
MIME-Version: 1.0
In-Reply-To: <78568af11001281435l782926e5g4c180668ae0b7f8c@mail.gmail.com>
References: <7c962aed1001281238g2c49d660pf47d23fe90432886@mail.gmail.com>
	 <78568af11001281242i3419dda8ke7fc082d980df14c@mail.gmail.com>
	 <4aa34eb71001281427j69768786o755bc931ebf72245@mail.gmail.com>
	 <78568af11001281435l782926e5g4c180668ae0b7f8c@mail.gmail.com>
Date: Thu, 28 Jan 2010 15:08:56 -0800
Message-ID: <c93bd2771001281508q5edfe4e1v14e020549e606001@mail.gmail.com>
Subject: Re: Discussion: Move contribs out of hbase?
From: Joydeep Sarma <jsensarma@gmail.com>
To: hbase-dev@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

fwiw - i agree with Dhruba's take. i can't imagine (for example)
hadoop shipping without hadoop-streaming (which is contrib). and it's
awesome that hadoop changes that break streaming are caught early and
often.

it makes sense this analog doesn't apply to all contrib projects -
perhaps they shouldn't be in contrib to begin with? alternatively - to
extend Dhruba's idea - one could break up contrib projects into those
that are critical (like streaming would be to hadoop and i imagine the
replication stuff would be to hbase) - and those that are non-critical
(and not build these by default).

the only reason to have non-critical contrib projects in the codebase
seems to be to give young/new projects exposure. i am not sure whether
this is a good enough reason - perhaps those with some historical
perspective have better idea on this.


On Thu, Jan 28, 2010 at 2:35 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> Part of the problem with contrib is fixing the deeper problems during
> major version changes (eg: new file format in 0.20) requires
> specialized knowledge that only the contrib author has. Then they must
> write a patch, submit it to a hbase committer, etc. =A0If the contrib
> was in a non-ASF repo, the author could fix it much faster.
>
> That's my theory at least?
>
> On Thu, Jan 28, 2010 at 2:27 PM, Dhruba Borthakur <dhruba@gmail.com> wrot=
e:
>> Hi stack,
>>
>> I have seen this issue crop up in the HDFS =A0repository too.
>>
>> The issue that you have brought up is that code changes to core sometime=
s
>> needs lots of changes to contrib to make it compile. This is a big probl=
em,
>> especially when all developers are not intricately familiar with each an=
d
>> every contrib module. However, the big plus is that when a HBase release=
 is
>> made, it has all the contrib modules in it, and it makes it easier for a
>> user to start using any of those contrib modules right away. Can we get =
the
>> best of both worlds if we do something like this:
>>
>> =A0Contribs are not first-class citizens. The build script can be setup =
to
>> ignore build failures in contrib modules. The hope is that when a releas=
e is
>> about to be cut, contrib owners will submit fixes to any build problems =
if
>> they want their contrib to be shipped as part of the Hbase release. will
>> this solve your problem?
>>
>> thanks,
>> dhruba
>>
>>
>>
>> On Thu, Jan 28, 2010 at 12:42 PM, Ryan Rawson <ryanobjc@gmail.com> wrote=
:
>>
>>> +1
>>>
>>> I think when we are ready, we should publish the replication as a
>>> contrib up on github. I really like github, its way better than any
>>> other system I've seen (in part due to git of course).
>>>
>>> I do like the notion of officially released contribs that are included
>>> at specific version numbers with the hbase release. Hbase core 0.21.0
>>> with TH, ITH at version 0.1 or whatever.
>>>
>>> -ryan
>>>
>>> On Thu, Jan 28, 2010 at 12:38 PM, Stack <stack@duboce.net> wrote:
>>> > I'd like to start a discussion on moving src/contrib out of hbase.
>>> > Keep reading if you have an opinion.
>>> >
>>> > I'd like to suggest that we undo the notion of hbase contribs for the
>>> > following reasons:
>>> >
>>> > + In my experience, they present a friction on changes to core as any
>>> > significant core change tends to ripple down into the contribs whethe=
r
>>> > its code or infrastructure/build changes. =A0Usually what happens the=
n
>>> > is a non-expert in the contrib code is making edits -- often radical
>>> > -- to code they are not completely up on and are a little frustrated
>>> > that they have to do it. =A0Bad.
>>> > + A few of our contribs are maintained by non-committers. =A0This mea=
ns
>>> > it takes a committers time getting in updates. =A0The owner is at mer=
cy
>>> > of the committer making wanted changes. =A0The committer is consumed
>>> > reviewing and making update. =A0This indirection hurts at both ends (=
We
>>> > could discuss making contrib owners committers on their contrib only
>>> > but that'd be a bit of bureaucratic nightmare and a burden on the
>>> > hadoop pmc to vote on granting access to a subprojects, contrib. =A0I=
ts
>>> > tough enough getting hadoop pmc to vote on hbase committers. =A0There=
 is
>>> > no precedent in other project, to my knowledge).
>>> > + Contribs and core evolve at different rates. =A0They should not be
>>> > constrained by core release schedule (or the opposite, core should no=
t
>>> > be held up because fixes in contrib are wanting).
>>> >
>>> > I suggest that current contribs be moved out of hbase up to standalon=
e
>>> > github or google code projects (witness how hbql does it). =A0Previou=
s
>>> > to our move to Ivy (and possibly soon, Maven), asking contribs be
>>> > standalone was a pain as they'd have to check in hbase jars and all o=
f
>>> > dependencies and then move these forward over time. =A0Now that we ar=
e
>>> > Ivy-ized, contribs just need write a bit of ivy.xml and it'll take
>>> > care of pulling dependencies.
>>> >
>>> > I'd imagine support for contribs would go on as it does now with
>>> > queries up on hbase mailing list and help out on IRC. =A0We'd give
>>> > contribs first-class billing up on home page. =A0Popular contribs mig=
ht
>>> > run their own mailing lists, etc...
>>> >
>>> > We should chat but some contribs should be pulled up into core.
>>> > Thrift was core. =A0Talk was to move current thrift update out to
>>> > contrib. =A0I think now it should just stay in core. =A0Stargate perh=
aps
>>> > should come up into core?
>>> >
>>> > What do folks think?
>>> > St.Ack
>>> >
>>> > P.S. I've fostered the contrib notion in the past. =A0I've since had =
a
>>> > change of heart. =A0Please pardon my flip.
>>> >
>>>
>>
>>
>>
>> --
>> Connect to me at http://www.facebook.com/dhruba
>>
>