hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: how to use RegionCoprocessorEnvironment getSharedData() to share data among coprocessors?
Date Thu, 16 Apr 2015 03:05:06 GMT
I looked in Phoenix codebase for usage of getSharedData() and found one in:

phoenix-core/src/main/java/org/apache/phoenix/hbase/index/parallel/ThreadPoolManager.java

   return getExecutor(builder, env.getSharedData());

You can refer to the above class as an example.

FYI

On Wed, Apr 15, 2015 at 6:03 PM, Liu, Ming (HPIT-GADSC) <ming.liu2@hp.com>
wrote:

> Hi, Ted,
>
> I am not sure. I am a C programmer, only know a very little java. It seems
> to me that the two coprocessors have to 'extends' from two different base
> classes for Endpoint and Observer. Java seems not be able to do this.
> I am just trying to study coprocessor, there is no solid production
> requirement to do this sharing thing. Coprocessor was introduced from 0.92,
> and no one need this before, so I am thinking this is not something
> important.
> Thanks always for your help! You are very kind to answer all my questions
> in this maillist :-)
> I will try to read the getSharedData code to further understand and
> feedback here if I can have some more findings or any other good method to
> shared data in a simple way.
>
> Thanks,
> Ming
>
> -----Original Message-----
> From: Ted Yu [mailto:yuzhihong@gmail.com]
> Sent: Thursday, April 16, 2015 5:16 AM
> To: user@hbase.apache.org
> Subject: Re: how to use RegionCoprocessorEnvironment getSharedData() to
> share data among coprocessors?
>
> Can you implement the Observer coprocessor and the Endpoint coprocessor in
> one class ?
>
> Cheers
>
> On Wed, Apr 15, 2015 at 9:00 AM, Liu, Ming (HPIT-GADSC) <ming.liu2@hp.com>
> wrote:
>
> > Thank you Ted,
> >
> > I am using 0.98.11.  I do read the first example, but I don't find the
> > second one before, I will try to read it, but it seems too complex for
> > me
> > :-)
> > I read from
> > http://hadoop-hbase.blogspot.com/2012/10/coprocessor-access-to-hbase-i
> > nternals.html That : "This shared data is per coprocessor class and
> > per regionserver."
> > So it means to me that hbase cannot share between two different
> > coprocessors like an endpoint and an observer by using this
> > sharedData, since there are two different classes in my case. I use
> > Zookeeper to share data for now, but I was told not to depend on
> > ZooKeeper too much, no idea why. Is there any other good way I can use
> > to share data among different coprocessors?
> >
> > Thanks,
> > Ming
> > -----Original Message-----
> > From: Ted Yu [mailto:yuzhihong@gmail.com]
> > Sent: Wednesday, April 15, 2015 8:25 PM
> > To: user@hbase.apache.org
> > Subject: Re: how to use RegionCoprocessorEnvironment getSharedData()
> > to share data among coprocessors?
> >
> > Which hbase release are you using ?
> >
> > Please take a look at the following tests for example of using
> > getSharedData() :
> >
> >
> > hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/examp
> > le/ZooKeeperScanPolicyObserver.java
> >
> > hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCop
> > rocessorInterface.java
> >
> > Cheers
> >
> > On Tue, Apr 14, 2015 at 10:35 PM, Liu, Ming (HPIT-GADSC)
> > <ming.liu2@hp.com
> > >
> > wrote:
> >
> > > Hi, all,
> > >
> > > I am trying to learn how to share data between two coprocessors. I
> > > have one Observer coprocessor and one Endpoint coprocessor. In the
> > > observer, it overload the prePut/preDelete to maintain a counter.
> > > And I want the Endpoint coprocessor to read that counter and return
> > > to client caller. So I want to use the getSharedData() method in
> > > RegionCoprocessorEnvironment, but I cannot make it work. Could
> > > anybody
> > help me here?
> > >
> > > In the Observer Coprocessor :
> > > During start(), create the shared object "counter":
> > > -----------------------------------------------------------------
> > > public void start(CoprocessorEnvironment envi) throws IOException {
> > >   Env.getSharedData().put("counter", new Long(0) ); //create the
> > > counter
> > > -----------------------------------------------------------------
> > >
> > > In the Endpoint coprocessor:
> > > During start(), try to read the shared "counter" , but failed.
> > > ------------------------------------------------------------------
> > > public void start(CoprocessorEnvironment envi) throws IOException {
> > >      LOG.info("The size of sharedData map is: " +
> > > envi.getSharedData().size() ); //try to get the counter
> > > ------------------------------------------------------------------
> > > Here it print 0, if I use
> > > evni.getSharedData().containsKey("counter"),
> > > it will return false.
> > >
> > > When creating table, I call addCoprocessor() method to add Observer
> > > first, then Endpoint coprocessor. I confirmed that by checking the
> > > hbase log file message. I only have one region for that table during
> > > the run. I confirmed by hbase shell status 'detailed' command.
> > >
> > > There is not much example I can find about how to use
> > > getSharedData(), could someone help me here? What is missing in my
> > > simple code? Thanks very much in advance!
> > >
> > > Thanks,
> > > Ming
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message