hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
Date Wed, 27 Sep 2017 19:50:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183142#comment-16183142
] 

Appy commented on HBASE-17732:
------------------------------

h4. Commit message
{noformat}
HBASE-17732 Coprocessor Design Improvements

    ------------------------------------------------------
    TL;DR
    ------------------------------------------------------
    We are moving from Inheritence
    - Observer *is* Coprocessor
    - FooService *is* CoprocessorService
    To Composition
    - Coprocessor *has* Observer
    - Coprocessor *has* Service

    ------------------------------------------------------
    Design Changes
    ------------------------------------------------------
    - Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
      WALCoprocessor
    - These new *Coprocessor interfaces have a get*Observer() function for each observer type
      supported by them.
    - Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces
will
      get it from the base interface.
    - Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager
doing its
      own trickery.
    - CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
      Deleted if can, else marked @VisibleForTesting.

    ------------------------------------------------------
    Backward Compatibility
    ------------------------------------------------------
    - Old coprocessors implementing *Observer won't get loaded (no backward compatibility
guarantees).
    - Third party coprocessors only implementing Coprocessor will not get loaded (just like
Observers).
    - Old coprocessors implementing CoprocessorService (for master/region host)
      /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
    - Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
    - Note that if a coprocessor implements both observer and service in same class, its service
      component will continue to work but it's observer component won't work.

    ------------------------------------------------------
    Notes
    ------------------------------------------------------
    Did a side-by-side comparison of CPs in master and after patch. These coprocessors which
were just
    CoprocessorService earlier, needed a home in some coprocessor in new design. For most
it was clear
    since they were using a particular type of environment. Some were tricky.

    - JMXListener - MasterCoprocessor and RSCoprocessor (because jmx listener makes sense
for
      processes?)
    - RSGroupAdminEndpoint --> MasterCP
    - VisibilityController -> MasterCP and RegionCP

    These were converted to RegionCoprocessor because they were using RegionCoprocessorEnvironment
    which can only come from a RegionCPHost.
    - AggregateImplementation
    - BaseRowProcessorEndpoint
    - BulkDeleteEndpoint
    - Export
    - RefreshHFilesEndpoint
    - RowCountEndpoint
    - MultiRowMutationEndpoint
    - SecureBulkLoadEndpoint
    - TokenProvider
{noformat}

> Coprocessor Design Improvements
> -------------------------------
>
>                 Key: HBASE-17732
>                 URL: https://issues.apache.org/jira/browse/HBASE-17732
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Appy
>            Assignee: Appy
>            Priority: Critical
>             Fix For: 2.0.0-alpha-4
>
>         Attachments: HBASE-17732.master.001.patch, HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
HBASE-17732.master.004.patch, HBASE-17732.master.005.patch, HBASE-17732.master.006.patch,
HBASE-17732.master.007.patch, HBASE-17732.master.008.patch, HBASE-17732.master.009.patch,
HBASE-17732.master.010.patch, HBASE-17732.master.011.patch, HBASE-17732.master.012.patch,
HBASE-17732.master.013.patch, HBASE-17732.master.014.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e. {{interface CoprocessorEnvironment<C
extends Coprocessor>}}*
>   ** Enables us to load only relevant coprocessors in hosts. Right now each type of host
loads all types of coprocs and it's only during execOperation that it checks if the coproc
is of correct type i.e. XCoprocessorHost will load XObserver, YObserver, and all others, and
will check in execOperation if {{coproc instanceOf XObserver}} and ignore the rest.
>   ** Allow sharing of a bunch functions/classes which are currently duplicated in each
host. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new classes and
and old observers*
>   ** The real gold here is, moving forward, we'll be able to break down giant everything-in-one
observers (masterobserver has 100+ functions) into smaller, more focused observers. These
smaller observer can then have different compat guarantees!!
> Here's a more detailed design doc: https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message