hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)
Date Fri, 16 Dec 2011 19:43:39 GMT
Jonathan:
If the feature is very big and cannot be partitioned, I agree with what you
said.

I can think of few examples of such features. Take a look at HBASE-2856,
HBASE-451, etc. They got completed without resorting to feature branch.

Cheers

On Fri, Dec 16, 2011 at 10:43 AM, Jonathan Hsieh <jon@cloudera.com> wrote:

> I'll make a few points and then leave it to you all to decide on how
> to proceed.  (Ideally, Lui Jia would chime in as well!)
>
> I agree about the perceived importance of being in an asf repo, which
> is why being a branch in the asf repo seems like an appealing approach
> -- it addresses the desire to be inclusive, and to address the
> concerns of more conservative members of the community.
>
> I'd argue that difficult isn't the same as impossible -- other
> projects have managed branches just fine.  Hadoop is an example that
> has had many branches (HA, append, security, etc) that
> for-better-or-for-worse eventually were or will be merged or
> abandoned.
>
> A feature branch allows the patch to get in which allows other folks
> to work on the patch.  Instead of a single mega-patch by a single
> person, the basics could be committed in the branch and follow on work
> could be broken up into manageably sized, quicker to review patches
> done by multiple people.  When it is ready, the merge happens.
> Admittedly I've only done merges in github-land, but since most of us
> I assume use git, the merge could happen there and could potentially
> just be applied to svn as a series of patches on mainline trunk when
> it is time.
>
> The feature branches that have the most merge pains ones that have
> cross-cutting changes.  In the particular case of HBASE-4120, the code
> only touches 3-4 existing files (RPC related) so I'd think the merge
> pain is relatively low (unless we change the rpc servers anytime
> soon).
>
> Jon.
>
> On Dec 14, 2011, at 12:13, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > I agree with Andy.
> >
> > Maintaining a patch that cleanly applies to TRUNK is a lot easier.
> >
> > Cheers
> >
> > On Wed, Dec 14, 2011 at 11:41 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> We have discussed feature branches in the past, and even attempted it
> once
> >> or twice (if I recall correctly, both with coprocessors and security),
> but
> >> Subversion merge support is lacking. I assert ASF tooling is suboptimal
> for
> >> managing a long lived feature branch. I'm happy to be properly educated
> if
> >> you disagree.
> >>
> >> Otherwise, this leads the discussion to external hosting of feature
> >> branches under development, where there is more appropriate tooling,
> such
> >> as up on GitHub.
> >>
> >> Trend put up our security work on a public GitHub repository. I
> routinely
> >> advertised it in any security related discussion. However, it was
> ignored
> >> by the community: As far as I know, nobody ever checked it out and tried
> >> it. Only code in the ASF repo seemed to matter.
> >>
> >> Best regards,
> >>
> >>
> >>   - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >>
> >> ----- Original Message -----
> >>> From: Jonathan Hsieh <jon@cloudera.com>
> >>> To: dev@hbase.apache.org
> >>> Cc: Andrew Purtell <apurtell@apache.org>
> >>> Sent: Tuesday, December 13, 2011 11:57 AM
> >>> Subject: Re: Code review request for hbase-4120 table priority
> >>>
> >>> Note: I've only done a quick look at the jira and the code.  The high
> >> level
> >>> design document/approach seems reasonable and I think most agree that
> >> this
> >>> is a useful feature and that a lot of effort has gone into it.
> >>>
> >>> The feature is off by default -- I can see one main difference in this
> >>> situation compared to other major newish generally-considered
> >> experimental
> >>> or incomplete features (replication, off-heap slab cache, online schema
> >>> changes).  This feature doesn't have one of the current HBase
> committers
> >>> using/testing it in their production  environments or in their test
> >>> environment.
> >>>
> >>> This seems perfect for *a feature branch* as we talked briefly about at
> >> the
> >>> Pow-wow.  There seem to be some problems identified that will result in
> >>> follow on issues (races mentioned).  Using a branch would:
> >>> * make it available at apache allows devs to test it
> >>> * allows a committer who is championing this to test it by using it
> more
> >>> and to iron out glaring problems in environment,
> >>> * encourages and shepards the contributor allowing them to justify
> >>> continued effort,
> >>> * allows all of us to defer the decision to fold the feature into 0.94
> >> (or
> >>> 0.96, or later) when more folks are familiar or comfortable with it.
> >>>
> >>> Who knows, maybe some of the TaoBao folks will eventually become
> >> committers.
> >>>
> >>> Jon.
> >>>
> >>> On Tue, Dec 13, 2011 at 12:42 AM, <yuzhihong@gmail.com> wrote:
> >>>
> >>>> Thanks for the suggestion, Lars.
> >>>> The original scope for 4120 is bigger than the latest patch which only
> >>>> covers table priorities.
> >>>>
> >>>> Let's perform more reviews for the current patch. We can create more
> >>>> subtasks for the umbrella feature.
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>>
> >>>> On Dec 12, 2011, at 11:23 PM, lars hofhansl <lhofhansl@yahoo.com>
> >>> wrote:
> >>>>
> >>>>> While I haven't looked (in depth) at the patch, yet, this is
> >>> definitely
> >>>> a feature that will be extremely helpful
> >>>>> for Salesforce's multitenant architecture to isolate tenants and
> >>>> services from each other.
> >>>>>
> >>>>> While we don't have HBase in our production data centers, yet
> >>> (working
> >>>> on it), I am certain that we will use this feature
> >>>>> eventually.
> >>>>>
> >>>>> Would it help to break the patch into multiple smaller patches?
> >>>>>
> >>>>> Off the bat I think of:
> >>>>> 1. the grouping logic
> >>>>> 2. regionserver configuration (caching, etc) per group
> >>>>> 3. table priorities
> >>>>> 4. etc... (folks who have actually looked at the patch can probably
> >>>> identify better demarcations between the aspects of this change.)
> >>>>>
> >>>>> That would certainly make it more manageable for me - personally
- to
> >>>> review the code.
> >>>>>
> >>>>> -- Lars
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>> From: Todd Lipcon <todd@cloudera.com>
> >>>>> To: dev@hbase.apache.org; Andrew Purtell <apurtell@apache.org>
> >>>>> Cc:
> >>>>> Sent: Monday, December 12, 2011 4:55 PM
> >>>>> Subject: Re: Code review request for hbase-4120 table priority
> >>>>>
> >>>>> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell
> >>> <apurtell@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> HBase as a project should not have as a criteria for inclusion
of
> >>> some
> >>>> feature that Cloudera and SU and Facebook run it. Core managed to
> >> escape
> >>>> Yahoo. Let's not run history in reverse here in HBase land. And,
> >>> actually,
> >>>> this makes it worse, because the the occurrence that a number of core
> >> HBase
> >>>> users (multiple) will all need something is substantially less likely
> >> than
> >>>> if one might find it useful; or, maybe, only users outside of those
> >> with
> >>>> such self-appointed attitude, yet perhaps a community multiples in
> >> size of
> >>>> "core users".
> >>>>>
> >>>>> It's not about Cloudera/SU/FB - it's about code that will be
> >>> supported
> >>>>> by people who are committed to the project. TrendMicro certainly
fits
> >>>>> the bill. I of course mean no offense to Lu Jia, but neither he
nor
> >>>>> Taobao has made continued contributions in the past - just one other
> >>>>> bug fix beyond the HBASE-4120 project.
> >>>>>
> >>>>> If we have a few of the core people committed to running this in
> >>>>> production and supporting it in the future, I'm all for it (just
> >>> like
> >>>>> I am +1 on security). I just want to avoid repeating mistakes like
> >> the
> >>>>> Avro server which isn't really supported despite being in our
> >>>>> codebase. (You'll note this was a Cloudera contribution but from
a
> >>>>> contributor who was doing this in his spare time rather than part
of
> >>>>> job responsibilities, and we have never run it in production
> >>>>> scenarios)
> >>>>>
> >>>>> I am consistently conservative on what goes into the project because
> >>>>> we have to stand behind what we release. I certainly don't think
> >>> _all_
> >>>>> core people should find every feature useful (eg REST and Thrift
are
> >>>>> examples of some things which are useless to many but I think make
> >>>>> sense). But if _no_ core people see a feature as a requirement then
> >>>>> I'd rather let it bake until we have many people requesting it.
> >>>>> Otherwise people download HBase, try out these "fringe"
> >>> features, and
> >>>>> get a bad taste in their mouth when they've bit-rot across several
> >>>>> versions of little usage.
> >>>>>
> >>>>> -Todd
> >>>>> --
> >>>>> Todd Lipcon
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> // Jonathan Hsieh (shay)
> >>> // Software Engineer, Cloudera
> >>> // jon@cloudera.com
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message