Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: <CALte62yJKPofxBGxYUrk0weiy5CgFgRauD9HgdQ4+Oax6xbmbg@mail.gmail.com>
References: <CAHnJtfOmEukXPHLEkh_Lc8f5Tgv08vKiHyf36tKAcv3H5Ta_6A@mail.gmail.com>
 <CALte62zjc+9edROUDBNw+XFfSpbOk6uHEcKWZwzn=tfgvnpeFA@mail.gmail.com>
 <CAHnJtfNPiOE8ZzOUQ7np5VpVTUb14Z3+SDkBLY_-QRUDxJ1VFg@mail.gmail.com>
 <CALte62xJ5es5YgwsqZagcMp1G=VDcx=aB2sUc0vk0-E6xwmfGg@mail.gmail.com>
 <CAHnJtfOmb1-MK4PkZ=jBmmLhdOVPP_8GN8nZfRsZNYMX8YFx1w@mail.gmail.com> <CALte62yJKPofxBGxYUrk0weiy5CgFgRauD9HgdQ4+Oax6xbmbg@mail.gmail.com>
From: Veerraju Tadimeti <tvvraju@gmail.com>
Date: Thu, 6 Jul 2017 15:59:51 -0500
Message-ID: <CAHnJtfPPaJ=c9rmwUDzYRxDh2UYtHS6yxUvn=50P6OscaU=Aqw@mail.gmail.com>
Subject: Re: scope of RegionCoprocessorEnvironment sharedData
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary="001a1140a99ada291c0553ac6501"
archived-at: Thu, 06 Jul 2017 20:59:59 -0000

--001a1140a99ada291c0553ac6501
Content-Type: text/plain; charset="UTF-8"

I would like to invoke postScannerNext() based on a scan attribute. When
two different scans are invoked simultaneously, one with attribute and one
without attribute ,  how can I invoke postScannerNext() based on the scan
attribute.

Currently i am doing like this

   - checking the scan attribute in preScannerOpen(),
      - if exists , set that attribute in sharedData,
   - check that variable in postScannerNext()
   - execute logic only if variable exists in sharedData

If first scan operation with attribute is requested, we set that in
sharedData in preScannerOpen(), meanwhile if second scan operation without
attribute is requested, I think , according to your reply,  the
postScannerNext() will get invoked for both the operations .

is this correct? if Yes, what is the alternative.

Thanks in Advance.


Thanks,
Raju,
(972)273-0155.

On Thu, Jul 6, 2017 at 1:39 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> If the two different scan operations are for the same region, same instance
> of coprocessor is involved.
>
> Otherwise, different instances of coprocessor are involved.
>
> On Thu, Jul 6, 2017 at 11:22 AM, Veerraju Tadimeti <tvvraju@gmail.com>
> wrote:
>
> > for two different scan operations , will same instance of coprocessor
> > executes?
> >
> > Thanks,
> > Raju,
> > (972)273-0155.
> >
> > On Thu, Jul 6, 2017 at 11:53 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > sharedData is created when coprocessor is loaded (per region).
> > >
> > > So as long as the coprocessor instance doesn't change, the data you put
> > in
> > > should be there.
> > >
> > > On Thu, Jul 6, 2017 at 9:20 AM, Veerraju Tadimeti <tvvraju@gmail.com>
> > > wrote:
> > >
> > > > Is sharedData per Operation request ?
> > > >
> > > > I mean for a scan operation, if we put variables in sharedData in
> > > > coprocessor, with a value,   will the same variables exists(with the
> > same
> > > > value) in sharedData in coprocessor for new scan operation ?
> > > >
> > > > Thanks,
> > > > Raju,
> > > > (972)273-0155.
> > > >
> > > > On Thu, Jul 6, 2017 at 10:38 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > sharedData is in RegionEnvironment which is created
> > > > > by RegionCoprocessorHost.
> > > > > RegionCoprocessorHost is per region.
> > > > >
> > > > > When variable is removed in postScannerClose(), it wouldn't affect
> > > > > sharedData
> > > > > in another region.
> > > > >
> > > > > FYI
> > > > >
> > > > > On Thu, Jul 6, 2017 at 7:14 AM, Veerraju Tadimeti <
> tvvraju@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > hi,
> > > > > >
> > > > > > I have few questions regarding scope of
> > > *RegionCoprocessorEnvironment*
> > > > > >  sharedData.
> > > > > >
> > > > > >
> > > > > >
> > > > > >    - *Is sharedData map is shared accross all instances
> > > simultaneously
> > > > ?*
> > > > > >       -  I am putting a variable in sharedData in
> preScannerOpen()
> > > > based
> > > > > on
> > > > > >       scan attribute,
> > > > > >       - check that variable exists in postScannerNext() then
> apply
> > > > logic,
> > > > > >       - remove the variable postScannerClose().
> > > > > >       - If data is in multiple regions, when one coprocessor
> > removes
> > > > > >       variable in postScannerClose(), will the variable is NULL
> for
> > > > > another
> > > > > >       region coprocessor in postScannerNext() ?
> > > > > >
> > > > > >
> > > > > >    - *    Is sharedData map is shared across all the client
> request
> > > > > >    operations ?*
> > > > > >
> > > > > > If a variable is set in sharedData for one client operation(say
> > > SCAN),
> > > > > will
> > > > > > the variable is available for another client operation(new SCAN)
> ?
> > > > > >
> > > > > >
> > > > > >    -  *Will the variables be garbage collected even if we dont
> > > > implement
> > > > > >    (removed variables in sharedData) postScannerClose() method*
> > > > > >
> > > > > >
> > > > > > Please find below the logic that I am using currently
> > > > > > *CODE: *
> > > > > >
> > > > > >     public RegionScanner
> > > > > > *preScannerOpen*(ObserverContext<RegionCoprocessorEnvironment>
> > > > > > e, Scan scan, RegionScanner s) throws IOException {
> > > > > >         byte[] useGetInPostScannerNext =
> scan.getAttribute(USE_GET_
> > > > > > OPERATION_IN_POST_SCANNER_NEXT);
> > > > > >         String useGetInPostScannerNextStr = Bytes.toString(
> > > > > > useGetInPostScannerNext);
> > > > > >         if (Boolean.parseBoolean(useGetInPostScannerNextStr)) {
> > > > > >             e.getEnvironment().getSharedData().put(USE_GET_
> > > > > > OPERATION_IN_POST_SCANNER_NEXT, useGetInPostScannerNextStr);
> > > > > >         }
> > > > > >         return super.preScannerOpen(e, scan, s);
> > > > > >     }
> > > > > >
> > > > > > @Override
> > > > > >     public boolean *postScannerNext*(final
> > > > > > ObserverContext<RegionCoprocessorEnvironment>
> > > > > > e,
> > > > > >             final InternalScanner s, final List<Result> results,
> > > final
> > > > > int
> > > > > > limit,
> > > > > >             final boolean hasMore) throws IOException {
> > > > > >         try {
> > > > > >
> > > > > >             if (canUseGetOperation(e)) {
> > > > > >
> > > > > >                //logic goes here
> > > > > >             }
> > > > > >         } catch (Exception ex) {
> > > > > >             logger.error("Exception in postScannerNext ", ex);
> > > > > >             throw new IOException(ex);
> > > > > >         }
> > > > > >         return hasMore;
> > > > > >     }
> > > > > >
> > > > > >     @Override
> > > > > >     public void
> > > > > > *postScannerClose*(ObserverContext<RegionCoprocessorEnvironment>
> > > > > > e, InternalScanner s) throws IOException {
> > > > > >         if (canUseGetOperation(e)) {
> > > > > >             e.getEnvironment().getSharedData().remove(USE_
> > > > > > GET_OPERATION_IN_POST_SCANNER_NEXT);
> > > > > >         }
> > > > > >         super.postScannerClose(e, s);
> > > > > >     }
> > > > > >
> > > > > >     private boolean *canUseGetOperation*(final
> > > > > > ObserverContext<RegionCoprocessorEnvironment>
> > > > > > e) {
> > > > > >         String useGetOperationInPostScannerNext = (String)
> > > > > > e.getEnvironment().getSharedData().get(USE_GET_
> > > > > OPERATION_IN_POST_SCANNER_
> > > > > > NEXT);
> > > > > >         return Boolean.parseBoolean(
> useGetOperationInPostScannerNe
> > > xt);
> > > > > >     }
> > > > > >
> > > > > > Thanks,
> > > > > > Raju,
> > > > > > (972)273-0155 <(972)%20273-0155>.
> > > > > >
> > > > >
> > > >
> > >
> >
>

--001a1140a99ada291c0553ac6501--