hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [DISCUSS] Coprocessor major design changes HBASE-17732
Date Tue, 26 Sep 2017 21:16:29 GMT
I like the direction.

Pieces of this approach have appeared on various JIRAs, perhaps even some I
might have filed, but of course not so well written. The general idea is to
move away user facing design from the Observer interface subtypes carrying
a huge number of hook points, even if default methods or abstract base
classes help, to small interfaces, small implementation code, ideally as
small and self contained as lambdas. Within the server implementation,
today's need to implement redundant methods in every host type is kind of
painful too.

Use of composition instead of inheritance - big +1. We wanted coprocessors
to allow feature implementors to compose extensions, a mixin approach to
server extension, and have fallen a little short of what we could do in the
API design itself.

Let me see if comments on the review in progress would be helpful.


On Thu, Sep 21, 2017 at 12:20 PM, Stack <stack@duboce.net> wrote:

> Folks ok w/ this proposed change for hbase2? Lots of disruption but lots of
> benefit. I'm game and will help land this in next few days unless
> objection?
> Thanks,
> S
>
> On Fri, Sep 8, 2017 at 4:29 PM, Stack <stack@duboce.net> wrote:
>
> > On Wed, Sep 6, 2017 at 5:07 PM, Apekshit Sharma <appy@cloudera.com>
> wrote:
> >
> >> Hi
> >>
> >> Links: HBASE-17732 <https://issues.apache.org/jira/browse/HBASE-17732>,
> >> Code
> >> Review <https://reviews.apache.org/r/62141/>
> >>
> >> With 2.0 beta 1 approaching, this seems to be the ripe time to make much
> >> needed coprocessor design changes. Here's a summary of what's changing
> >> (from jira description):
> >>
> >> The two main changes are:
> >>
> >>    - *Adding template for coprocessor type to CoprocessorEnvironment
> i.e.
> >>    interface CoprocessorEnvironment<C extends Coprocessor>*
> >>       - Enables us to load only relevant coprocessors in hosts. Right
> now
> >>       each type of host loads all types of coprocs and it's only during
> >>       execOperation that it checks if the coproc is of correct type i.e.
> >>       XCoprocessorHost will load XObserver, YObserver, and all others,
> >> and will
> >>       check in execOperation if coproc instanceOf XObserver and ignore
> >> the rest.
> >>       - Allow sharing of a bunch functions/classes which are currently
> >>       duplicated in each host. For eg. CoprocessorOperations,
> >>       CoprocessorOperationWithResult, execOperations().
> >>    - *Introduce 4 coprocessor classes and use composition between these
> >> new
> >>    classes and and old observers*
> >>       - The real gold here is, moving forward, we'll be able to break
> down
> >>       giant everything-in-one observers (masterobserver has 100+
> >> functions) into
> >>       smaller, more focused observers. These smaller observer can then
> >> have
> >>       different compat guarantees!!
> >>
> >>
> >> There's a more detailed analysis/design doc which discusses motivation
> >> behind the changes:
> >> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-
> >> rOpQmZuixCDua-LT9X4/edit
> >>
> >> Impact on third party:
> >> - Although it'll break every coprocessor out there, the change to make
> >> things work again will be very tiny. Look at the changes I had to do to
> >> Coprocessors in our tests. It's just few simple lines.
> >> - CoprocessorService will be backward compatible.
> >>
> >> Status: Almost code complete. Fixing last 2-3 tests and adding a test to
> >> ensure backward compatibility of CoprocessorService.
> >>
> >> ------------------------------------------------------------
> >> ------------------------------------
> >> Jira: https://issues.apache.org/jira/browse/HBASE-17732
> >> Code review: https://reviews.apache.org/r/62141/
> >> Doc:
> >> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-
> >> rOpQmZuixCDua-LT9X4/edit
> >> ------------------------------------------------------------
> >> ------------------------------------
> >>
> >> -- Appy
> >>
> >
> > Nice writeup. I make an attempt at summarizing benefit in a comment. Fix
> > if I have it wrong. Downside is total disruption (though argument is the
> > change to come up on new framework is small change) though we are having
> > total disruption anyways in 2.0 for Coprocessors.
> >
> > This refactor does nothing for long-standing issues like HBASE-15071
> > or HBASE-6992 ?
> >
> > Thanks Appy,
> > S
> >
> >
> >
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message