hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: dynamic RPC and coprocessor changes
Date Wed, 06 Oct 2010 15:55:15 GMT
> From: Ryan Rawson
> For what it's worth, the Result changes are just a few functions
> and represent a big bang for the buck in terms of lines of code
> added.  I fixed the build - sorry about that!

I wasn't trying to pick on these changes. You'll see a +1 from me on the jira because I agree
.. big bang for the buck. It's just that seeing changes like this and others (blooms, replication,
master changes, hfof, etc.) go in make rationalizations about 0.90 being a stability-only
release incongruent. 

Let's not forget 0.90 is in fact a major release that contains a number of new features. 0.89
has not been a "real" release. 0.90 is a new major release that has new features relative
to the last major release branch 0.20.x. 

It piques me that coprocessors have been largely deferred by everyone (I won't say "ignored")
and this is used as a reason to hold up including them in the upcoming release. Especially
as they do not change any core semantic. Except, arguably, the RPC changes may drift up to
that line, I grant that.

> From: Jon Gray
> I just don't think this is the kind of thing that gets nailed with
> two weeks of code reviews.  It needs to be used, exercised, and
> different use cases need to be applied to it.
> For example, the initial CP implementation underwent a huge amount
> of change once you guys started to make security work with it.
> That's just one use case.  Others will push it in new directions
> and will have different requirements.

I totally agree with this. However, as a project we have only been going through with deprecation
in one release and removal in the next for client side code. 

Our client side changes are very generic and flexible. 

On the server side, the interfaces indeed may change with experience, but I anticipate the
predominant type of change would be additions, as new users find they can't do something with
the existing hooks, not change or removal of existing methods. For example, we have not added
HLog hooking or looked at master extensibility (at least, how extending HLog plays with recovery).
But this can be added without impacting any users of existing interfaces.  

We have already put coprocessors up in a feature branch, up on GitHub (http://github.com/trendmicro/hbase).
I don't think it makes sense to do this in Apache SVN, because SVN is a lousy tool for maintaining
feature branches and nobody looks at them anyway. However we know that the majority of users
will not pull some arbitrary code from GitHub to use given there is an official ASF release
tarball available. That's not an unreasonable position on their part. For coprocessors to
have the widest exposure or use, they need to go in to a release. 

> From: Gary Helmling
> For coprocessors, sure there were quite a few changes from the
> initial patches to actually using it to implement security, but a
> lot of that amounted to paring it down to a core usable scope.
> It will doubtless not address all use cases, but I think that will
> remain true whether it's included in 0.90 or 0.92.

Or 0.91, or 0.93, or 0.94 ...

This is the very first cut of a new feature. 

I expect to branch out from this beachhead in multiple directions, new extensibility for new
use cases, some we know, some we haven't anticipated yet. I hope to introduce code weaving
for the reasons talked about up on jira under HBASE-2000 somewhere (I think 2001). 

But we have to start somewhere...

   - Andy


View raw message