hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joydeep Sarma <jsensa...@gmail.com>
Subject Re: Discussion: Move contribs out of hbase?
Date Thu, 28 Jan 2010 23:08:56 GMT
fwiw - i agree with Dhruba's take. i can't imagine (for example)
hadoop shipping without hadoop-streaming (which is contrib). and it's
awesome that hadoop changes that break streaming are caught early and

it makes sense this analog doesn't apply to all contrib projects -
perhaps they shouldn't be in contrib to begin with? alternatively - to
extend Dhruba's idea - one could break up contrib projects into those
that are critical (like streaming would be to hadoop and i imagine the
replication stuff would be to hbase) - and those that are non-critical
(and not build these by default).

the only reason to have non-critical contrib projects in the codebase
seems to be to give young/new projects exposure. i am not sure whether
this is a good enough reason - perhaps those with some historical
perspective have better idea on this.

On Thu, Jan 28, 2010 at 2:35 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> Part of the problem with contrib is fixing the deeper problems during
> major version changes (eg: new file format in 0.20) requires
> specialized knowledge that only the contrib author has. Then they must
> write a patch, submit it to a hbase committer, etc.  If the contrib
> was in a non-ASF repo, the author could fix it much faster.
> That's my theory at least?
> On Thu, Jan 28, 2010 at 2:27 PM, Dhruba Borthakur <dhruba@gmail.com> wrote:
>> Hi stack,
>> I have seen this issue crop up in the HDFS  repository too.
>> The issue that you have brought up is that code changes to core sometimes
>> needs lots of changes to contrib to make it compile. This is a big problem,
>> especially when all developers are not intricately familiar with each and
>> every contrib module. However, the big plus is that when a HBase release is
>> made, it has all the contrib modules in it, and it makes it easier for a
>> user to start using any of those contrib modules right away. Can we get the
>> best of both worlds if we do something like this:
>>  Contribs are not first-class citizens. The build script can be setup to
>> ignore build failures in contrib modules. The hope is that when a release is
>> about to be cut, contrib owners will submit fixes to any build problems if
>> they want their contrib to be shipped as part of the Hbase release. will
>> this solve your problem?
>> thanks,
>> dhruba
>> On Thu, Jan 28, 2010 at 12:42 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>>> +1
>>> I think when we are ready, we should publish the replication as a
>>> contrib up on github. I really like github, its way better than any
>>> other system I've seen (in part due to git of course).
>>> I do like the notion of officially released contribs that are included
>>> at specific version numbers with the hbase release. Hbase core 0.21.0
>>> with TH, ITH at version 0.1 or whatever.
>>> -ryan
>>> On Thu, Jan 28, 2010 at 12:38 PM, Stack <stack@duboce.net> wrote:
>>> > I'd like to start a discussion on moving src/contrib out of hbase.
>>> > Keep reading if you have an opinion.
>>> >
>>> > I'd like to suggest that we undo the notion of hbase contribs for the
>>> > following reasons:
>>> >
>>> > + In my experience, they present a friction on changes to core as any
>>> > significant core change tends to ripple down into the contribs whether
>>> > its code or infrastructure/build changes.  Usually what happens then
>>> > is a non-expert in the contrib code is making edits -- often radical
>>> > -- to code they are not completely up on and are a little frustrated
>>> > that they have to do it.  Bad.
>>> > + A few of our contribs are maintained by non-committers.  This means
>>> > it takes a committers time getting in updates.  The owner is at mercy
>>> > of the committer making wanted changes.  The committer is consumed
>>> > reviewing and making update.  This indirection hurts at both ends (We
>>> > could discuss making contrib owners committers on their contrib only
>>> > but that'd be a bit of bureaucratic nightmare and a burden on the
>>> > hadoop pmc to vote on granting access to a subprojects, contrib.  Its
>>> > tough enough getting hadoop pmc to vote on hbase committers.  There is
>>> > no precedent in other project, to my knowledge).
>>> > + Contribs and core evolve at different rates.  They should not be
>>> > constrained by core release schedule (or the opposite, core should not
>>> > be held up because fixes in contrib are wanting).
>>> >
>>> > I suggest that current contribs be moved out of hbase up to standalone
>>> > github or google code projects (witness how hbql does it).  Previous
>>> > to our move to Ivy (and possibly soon, Maven), asking contribs be
>>> > standalone was a pain as they'd have to check in hbase jars and all of
>>> > dependencies and then move these forward over time.  Now that we are
>>> > Ivy-ized, contribs just need write a bit of ivy.xml and it'll take
>>> > care of pulling dependencies.
>>> >
>>> > I'd imagine support for contribs would go on as it does now with
>>> > queries up on hbase mailing list and help out on IRC.  We'd give
>>> > contribs first-class billing up on home page.  Popular contribs might
>>> > run their own mailing lists, etc...
>>> >
>>> > We should chat but some contribs should be pulled up into core.
>>> > Thrift was core.  Talk was to move current thrift update out to
>>> > contrib.  I think now it should just stay in core.  Stargate perhaps
>>> > should come up into core?
>>> >
>>> > What do folks think?
>>> > St.Ack
>>> >
>>> > P.S. I've fostered the contrib notion in the past.  I've since had a
>>> > change of heart.  Please pardon my flip.
>>> >
>> --
>> Connect to me at http://www.facebook.com/dhruba

View raw message