hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anoop John <anoop.hb...@gmail.com>
Subject Re: scope of RegionCoprocessorEnvironment sharedData
Date Mon, 10 Jul 2017 05:57:39 GMT
Ya as Ted said, u are not getting Scan object in the postScannerNext
and so can not make use of the attribute in Scan within this hook.
Just setting the sharedData variable will cause issue with concurrent
scans. (As u imagine)

So I can think of solving this in 2 possible ways. (May be more ways possible)

1.  U keep a Map within ur CP impl.  You implement postScannerOpen
where u will get the ref to Scanner been created as well as the Scan.
If the Scan is having attribute, keep that scanner within ur Map.
During postScannerNext  check if the coming in scanner is there in ur
Map. If so that means this is the one where u can do the action.
Also dont forget to implement postScannerClose and remove that scanner
from the Map.   Here u might have some perf penalty as u have to add
and get from Map which has to be a concurrent map too.

Another way

2. Create a custom scanner implementing RegionScanner.   The new one
has to take an original Region Scanner and just delegate the calls. On
postScannerOpen, u will get the original scanner been created and u
can just wrap it with ur new scanner object. ( If the Scan object is
having required attribute)..  In postScannerNext() u can check for ur
own RegionScanner type and if so u can do action.


-Anoop-


On Sat, Jul 8, 2017 at 9:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>             if (canUseGetOperation(e)) {
>                //logic goes here
>
> Does your Get target the same region being scanned ?
> If not, issuing the Get is not advised since the other region may be hosted
> on different region server.
>
> Cheers
>
> On Thu, Jul 6, 2017 at 7:14 AM, Veerraju Tadimeti <tvvraju@gmail.com> wrote:
>
>> hi,
>>
>> I have few questions regarding scope of *RegionCoprocessorEnvironment*
>>  sharedData.
>>
>>
>>
>>    - *Is sharedData map is shared accross all instances simultaneously ?*
>>       -  I am putting a variable in sharedData in preScannerOpen() based on
>>       scan attribute,
>>       - check that variable exists in postScannerNext() then apply logic,
>>       - remove the variable postScannerClose().
>>       - If data is in multiple regions, when one coprocessor removes
>>       variable in postScannerClose(), will the variable is NULL for another
>>       region coprocessor in postScannerNext() ?
>>
>>
>>    - *    Is sharedData map is shared across all the client request
>>    operations ?*
>>
>> If a variable is set in sharedData for one client operation(say SCAN), will
>> the variable is available for another client operation(new SCAN) ?
>>
>>
>>    -  *Will the variables be garbage collected even if we dont implement
>>    (removed variables in sharedData) postScannerClose() method*
>>
>>
>> Please find below the logic that I am using currently
>> *CODE: *
>>
>>     public RegionScanner
>> *preScannerOpen*(ObserverContext<RegionCoprocessorEnvironment>
>> e, Scan scan, RegionScanner s) throws IOException {
>>         byte[] useGetInPostScannerNext = scan.getAttribute(USE_GET_
>> OPERATION_IN_POST_SCANNER_NEXT);
>>         String useGetInPostScannerNextStr = Bytes.toString(
>> useGetInPostScannerNext);
>>         if (Boolean.parseBoolean(useGetInPostScannerNextStr)) {
>>             e.getEnvironment().getSharedData().put(USE_GET_
>> OPERATION_IN_POST_SCANNER_NEXT, useGetInPostScannerNextStr);
>>         }
>>         return super.preScannerOpen(e, scan, s);
>>     }
>>
>> @Override
>>     public boolean *postScannerNext*(final
>> ObserverContext<RegionCoprocessorEnvironment>
>> e,
>>             final InternalScanner s, final List<Result> results, final int
>> limit,
>>             final boolean hasMore) throws IOException {
>>         try {
>>
>>             if (canUseGetOperation(e)) {
>>
>>                //logic goes here
>>             }
>>         } catch (Exception ex) {
>>             logger.error("Exception in postScannerNext ", ex);
>>             throw new IOException(ex);
>>         }
>>         return hasMore;
>>     }
>>
>>     @Override
>>     public void
>> *postScannerClose*(ObserverContext<RegionCoprocessorEnvironment>
>> e, InternalScanner s) throws IOException {
>>         if (canUseGetOperation(e)) {
>>             e.getEnvironment().getSharedData().remove(USE_
>> GET_OPERATION_IN_POST_SCANNER_NEXT);
>>         }
>>         super.postScannerClose(e, s);
>>     }
>>
>>     private boolean *canUseGetOperation*(final
>> ObserverContext<RegionCoprocessorEnvironment>
>> e) {
>>         String useGetOperationInPostScannerNext = (String)
>> e.getEnvironment().getSharedData().get(USE_GET_OPERATION_IN_POST_SCANNER_
>> NEXT);
>>         return Boolean.parseBoolean(useGetOperationInPostScannerNext);
>>     }
>>
>> Thanks,
>> Raju,
>> (972)273-0155 <(972)%20273-0155>.
>>

Mime
View raw message