hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: [DISCUSS] Coprocessor major design changes HBASE-17732
Date Thu, 21 Sep 2017 19:20:06 GMT
Folks ok w/ this proposed change for hbase2? Lots of disruption but lots of
benefit. I'm game and will help land this in next few days unless objection?
Thanks,
S

On Fri, Sep 8, 2017 at 4:29 PM, Stack <stack@duboce.net> wrote:

> On Wed, Sep 6, 2017 at 5:07 PM, Apekshit Sharma <appy@cloudera.com> wrote:
>
>> Hi
>>
>> Links: HBASE-17732 <https://issues.apache.org/jira/browse/HBASE-17732>,
>> Code
>> Review <https://reviews.apache.org/r/62141/>
>>
>> With 2.0 beta 1 approaching, this seems to be the ripe time to make much
>> needed coprocessor design changes. Here's a summary of what's changing
>> (from jira description):
>>
>> The two main changes are:
>>
>>    - *Adding template for coprocessor type to CoprocessorEnvironment i.e.
>>    interface CoprocessorEnvironment<C extends Coprocessor>*
>>       - Enables us to load only relevant coprocessors in hosts. Right now
>>       each type of host loads all types of coprocs and it's only during
>>       execOperation that it checks if the coproc is of correct type i.e.
>>       XCoprocessorHost will load XObserver, YObserver, and all others,
>> and will
>>       check in execOperation if coproc instanceOf XObserver and ignore
>> the rest.
>>       - Allow sharing of a bunch functions/classes which are currently
>>       duplicated in each host. For eg. CoprocessorOperations,
>>       CoprocessorOperationWithResult, execOperations().
>>    - *Introduce 4 coprocessor classes and use composition between these
>> new
>>    classes and and old observers*
>>       - The real gold here is, moving forward, we'll be able to break down
>>       giant everything-in-one observers (masterobserver has 100+
>> functions) into
>>       smaller, more focused observers. These smaller observer can then
>> have
>>       different compat guarantees!!
>>
>>
>> There's a more detailed analysis/design doc which discusses motivation
>> behind the changes:
>> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-
>> rOpQmZuixCDua-LT9X4/edit
>>
>> Impact on third party:
>> - Although it'll break every coprocessor out there, the change to make
>> things work again will be very tiny. Look at the changes I had to do to
>> Coprocessors in our tests. It's just few simple lines.
>> - CoprocessorService will be backward compatible.
>>
>> Status: Almost code complete. Fixing last 2-3 tests and adding a test to
>> ensure backward compatibility of CoprocessorService.
>>
>> ------------------------------------------------------------
>> ------------------------------------
>> Jira: https://issues.apache.org/jira/browse/HBASE-17732
>> Code review: https://reviews.apache.org/r/62141/
>> Doc:
>> https://docs.google.com/document/d/1PBEnaqyJeiHvALFswF_yl81-
>> rOpQmZuixCDua-LT9X4/edit
>> ------------------------------------------------------------
>> ------------------------------------
>>
>> -- Appy
>>
>
> Nice writeup. I make an attempt at summarizing benefit in a comment. Fix
> if I have it wrong. Downside is total disruption (though argument is the
> change to come up on new framework is small change) though we are having
> total disruption anyways in 2.0 for Coprocessors.
>
> This refactor does nothing for long-standing issues like HBASE-15071
> or HBASE-6992 ?
>
> Thanks Appy,
> S
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message