hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19001) Remove the hooks in RegionObserver which are designed to construct a StoreScanner which is marked as IA.Private
Date Wed, 18 Oct 2017 02:34:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208728#comment-16208728

Duo Zhang commented on HBASE-19001:

On the table if they set, the issue is this will be honored at the time of actual scan from
client side also. So the client side scan objects also has to specify this 'n' max versions.
There is no such problem. The default behavior is reading the latest version only. If you
want to read more versions you need to set it explicitly on the Scan or Get object. And for
the CP users, you must inject every where to filter out the cells which is only supposed to
use at an intermediate state, so if you do not specify it on the family config, then you need
to manually reset the versions every where also. This just adds more complexity I think.

Anyway, I think we can expose some configs in ScanInfo object to CP users, at least changing
versions and ttl is safe I think. And the good things maybe that, a end user can get the correct
actual versions information directly from the metadata of the Table, and for CP implementation
they do not need to use another option to indicate the actual versions. And when implementing
the wrapper InternalScanner, I found that it is really a pain to do filtering by versions(TTL
is simple) as we need to consider column change and row change and the cells returned by one
next call may not be a full row. And also, preStoreScannerOpen maybe a good place to inject
some simple logics as it is on the path of all read operations, such as Get, Scan,checkAndXXX,
and even increment. If CP users only want to change the versions or TTL then injecting here
is much easier than injecting every individual operations.

I could do this in a follow on issue, along with the constructor changes of StoreScanner(remove
the Optional parameter!).


> Remove the hooks in RegionObserver which are designed to construct a StoreScanner which
is marked as IA.Private
> ---------------------------------------------------------------------------------------------------------------
>                 Key: HBASE-19001
>                 URL: https://issues.apache.org/jira/browse/HBASE-19001
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Coprocessors
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0-alpha-4
>         Attachments: HBASE-19001-v1.patch, HBASE-19001-v2.patch, HBASE-19001.patch
> There are three methods here
> {code}
> KeyValueScanner preStoreScannerOpen(ObserverContext<RegionCoprocessorEnvironment>
>       Store store, Scan scan, NavigableSet<byte[]> targetCols, KeyValueScanner
s, long readPt)
>       throws IOException;
> InternalScanner preFlushScannerOpen(ObserverContext<RegionCoprocessorEnvironment>
>       Store store, List<KeyValueScanner> scanners, InternalScanner s, long readPoint)
>       throws IOException;
> InternalScanner preCompactScannerOpen(ObserverContext<RegionCoprocessorEnvironment>
>       Store store, List<? extends KeyValueScanner> scanners, ScanType scanType,
long earliestPutTs,
>       InternalScanner s, CompactionLifeCycleTracker tracker, CompactionRequest request,
>       long readPoint) throws IOException;
> {code}
> For the flush and compact ones, we've discussed many times, it is not safe to let user
inject a Filter or even implement their own InternalScanner using the store file scanners,
as our correctness highly depends on the complicated logic in SQM and StoreScanner. CP users
are expected to wrap the original InternalScanner(it is a StoreScanner anyway) in preFlush/preCompact
methods to do filtering or something else.
> For preStoreScannerOpen it even returns a KeyValueScanner which is marked as IA.Private...
This is less hurt but still, we've decided to not expose StoreScanner to CP users so here
this method is useless. CP users can use preGetOp and preScannerOpen method to modify the
Get/Scan object passed in to inject into the scan operation.

This message was sent by Atlassian JIRA

View raw message