Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CALte62zNtgz4Tuj2ZooNHMTA2SQxOPgaOwjdVT==V7v49GCrJw@mail.gmail.com>
References: <CAHnJtfOmEukXPHLEkh_Lc8f5Tgv08vKiHyf36tKAcv3H5Ta_6A@mail.gmail.com>
 <CALte62yWCaFuKGjwr1poAtMaaLmTPRtfeYigkxjMbLCcYQUNRw@mail.gmail.com>
 <CAOtJ30qdWQH9_WHKrQewLHSFZg1bsVWk28eA=a1iXiuHJcbzfg@mail.gmail.com> <CALte62zNtgz4Tuj2ZooNHMTA2SQxOPgaOwjdVT==V7v49GCrJw@mail.gmail.com>
From: Anoop John <anoop.hbase@gmail.com>
Date: Tue, 11 Jul 2017 18:35:40 +0530
Message-ID: <CAOtJ30qJYOJitzzx=s-wxYNxKdg5wiHn_PiqwajtzRr5kQhtOA@mail.gmail.com>
Subject: Re: scope of RegionCoprocessorEnvironment sharedData
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: text/plain; charset="UTF-8"
archived-at: Tue, 11 Jul 2017 13:05:55 -0000

Ya. It is the same RegionScanner impl in use only being passed.  Ya
the param type should have been RegionScanner  I guess. We made that
mistake!
-Anoop-

On Mon, Jul 10, 2017 at 8:37 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> The tricky part is that postScannerNext() passes InternalScanner parameter
> instead of RegionScanner.
>
> FYI
>
> On Sun, Jul 9, 2017 at 10:57 PM, Anoop John <anoop.hbase@gmail.com> wrote:
>
>> Ya as Ted said, u are not getting Scan object in the postScannerNext
>> and so can not make use of the attribute in Scan within this hook.
>> Just setting the sharedData variable will cause issue with concurrent
>> scans. (As u imagine)
>>
>> So I can think of solving this in 2 possible ways. (May be more ways
>> possible)
>>
>> 1.  U keep a Map within ur CP impl.  You implement postScannerOpen
>> where u will get the ref to Scanner been created as well as the Scan.
>> If the Scan is having attribute, keep that scanner within ur Map.
>> During postScannerNext  check if the coming in scanner is there in ur
>> Map. If so that means this is the one where u can do the action.
>> Also dont forget to implement postScannerClose and remove that scanner
>> from the Map.   Here u might have some perf penalty as u have to add
>> and get from Map which has to be a concurrent map too.
>>
>> Another way
>>
>> 2. Create a custom scanner implementing RegionScanner.   The new one
>> has to take an original Region Scanner and just delegate the calls. On
>> postScannerOpen, u will get the original scanner been created and u
>> can just wrap it with ur new scanner object. ( If the Scan object is
>> having required attribute)..  In postScannerNext() u can check for ur
>> own RegionScanner type and if so u can do action.
>>
>>
>> -Anoop-
>>
>>
>> On Sat, Jul 8, 2017 at 9:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> >             if (canUseGetOperation(e)) {
>> >                //logic goes here
>> >
>> > Does your Get target the same region being scanned ?
>> > If not, issuing the Get is not advised since the other region may be
>> hosted
>> > on different region server.
>> >
>> > Cheers
>> >
>> > On Thu, Jul 6, 2017 at 7:14 AM, Veerraju Tadimeti <tvvraju@gmail.com>
>> wrote:
>> >
>> >> hi,
>> >>
>> >> I have few questions regarding scope of *RegionCoprocessorEnvironment*
>> >>  sharedData.
>> >>
>> >>
>> >>
>> >>    - *Is sharedData map is shared accross all instances simultaneously
>> ?*
>> >>       -  I am putting a variable in sharedData in preScannerOpen()
>> based on
>> >>       scan attribute,
>> >>       - check that variable exists in postScannerNext() then apply
>> logic,
>> >>       - remove the variable postScannerClose().
>> >>       - If data is in multiple regions, when one coprocessor removes
>> >>       variable in postScannerClose(), will the variable is NULL for
>> another
>> >>       region coprocessor in postScannerNext() ?
>> >>
>> >>
>> >>    - *    Is sharedData map is shared across all the client request
>> >>    operations ?*
>> >>
>> >> If a variable is set in sharedData for one client operation(say SCAN),
>> will
>> >> the variable is available for another client operation(new SCAN) ?
>> >>
>> >>
>> >>    -  *Will the variables be garbage collected even if we dont implement
>> >>    (removed variables in sharedData) postScannerClose() method*
>> >>
>> >>
>> >> Please find below the logic that I am using currently
>> >> *CODE: *
>> >>
>> >>     public RegionScanner
>> >> *preScannerOpen*(ObserverContext<RegionCoprocessorEnvironment>
>> >> e, Scan scan, RegionScanner s) throws IOException {
>> >>         byte[] useGetInPostScannerNext = scan.getAttribute(USE_GET_
>> >> OPERATION_IN_POST_SCANNER_NEXT);
>> >>         String useGetInPostScannerNextStr = Bytes.toString(
>> >> useGetInPostScannerNext);
>> >>         if (Boolean.parseBoolean(useGetInPostScannerNextStr)) {
>> >>             e.getEnvironment().getSharedData().put(USE_GET_
>> >> OPERATION_IN_POST_SCANNER_NEXT, useGetInPostScannerNextStr);
>> >>         }
>> >>         return super.preScannerOpen(e, scan, s);
>> >>     }
>> >>
>> >> @Override
>> >>     public boolean *postScannerNext*(final
>> >> ObserverContext<RegionCoprocessorEnvironment>
>> >> e,
>> >>             final InternalScanner s, final List<Result> results, final
>> int
>> >> limit,
>> >>             final boolean hasMore) throws IOException {
>> >>         try {
>> >>
>> >>             if (canUseGetOperation(e)) {
>> >>
>> >>                //logic goes here
>> >>             }
>> >>         } catch (Exception ex) {
>> >>             logger.error("Exception in postScannerNext ", ex);
>> >>             throw new IOException(ex);
>> >>         }
>> >>         return hasMore;
>> >>     }
>> >>
>> >>     @Override
>> >>     public void
>> >> *postScannerClose*(ObserverContext<RegionCoprocessorEnvironment>
>> >> e, InternalScanner s) throws IOException {
>> >>         if (canUseGetOperation(e)) {
>> >>             e.getEnvironment().getSharedData().remove(USE_
>> >> GET_OPERATION_IN_POST_SCANNER_NEXT);
>> >>         }
>> >>         super.postScannerClose(e, s);
>> >>     }
>> >>
>> >>     private boolean *canUseGetOperation*(final
>> >> ObserverContext<RegionCoprocessorEnvironment>
>> >> e) {
>> >>         String useGetOperationInPostScannerNext = (String)
>> >> e.getEnvironment().getSharedData().get(USE_GET_
>> OPERATION_IN_POST_SCANNER_
>> >> NEXT);
>> >>         return Boolean.parseBoolean(useGetOperationInPostScannerNext);
>> >>     }
>> >>
>> >> Thanks,
>> >> Raju,
>> >> (972)273-0155 <(972)%20273-0155>.
>> >>
>>