hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu, Ming (HPIT-GADSC)" <ming.l...@hp.com>
Subject RE: how to use RegionCoprocessorEnvironment getSharedData() to share data among coprocessors?
Date Wed, 15 Apr 2015 16:00:34 GMT
Thank you Ted,

I am using 0.98.11.  I do read the first example, but I don't find the second one before,
I will try to read it, but it seems too complex for me :-) 
I read from http://hadoop-hbase.blogspot.com/2012/10/coprocessor-access-to-hbase-internals.html

That : "This shared data is per coprocessor class and per regionserver."
So it means to me that hbase cannot share between two different coprocessors like an endpoint
and an observer by using this sharedData, since there are two different classes in my case.
I use Zookeeper to share data for now, but I was told not to depend on ZooKeeper too much,
no idea why. Is there any other good way I can use to share data among different coprocessors?
 

Thanks,
Ming
-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Wednesday, April 15, 2015 8:25 PM
To: user@hbase.apache.org
Subject: Re: how to use RegionCoprocessorEnvironment getSharedData() to share data among coprocessors?

Which hbase release are you using ?

Please take a look at the following tests for example of using getSharedData() :

hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ZooKeeperScanPolicyObserver.java
hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java

Cheers

On Tue, Apr 14, 2015 at 10:35 PM, Liu, Ming (HPIT-GADSC) <ming.liu2@hp.com>
wrote:

> Hi, all,
>
> I am trying to learn how to share data between two coprocessors. I 
> have one Observer coprocessor and one Endpoint coprocessor. In the 
> observer, it overload the prePut/preDelete to maintain a counter. And 
> I want the Endpoint coprocessor to read that counter and return to 
> client caller. So I want to use the getSharedData() method in 
> RegionCoprocessorEnvironment, but I cannot make it work. Could anybody help me here?
>
> In the Observer Coprocessor :
> During start(), create the shared object "counter":
> -----------------------------------------------------------------
> public void start(CoprocessorEnvironment envi) throws IOException {
>   Env.getSharedData().put("counter", new Long(0) ); //create the 
> counter
> -----------------------------------------------------------------
>
> In the Endpoint coprocessor:
> During start(), try to read the shared "counter" , but failed.
> ------------------------------------------------------------------
> public void start(CoprocessorEnvironment envi) throws IOException {
>      LOG.info("The size of sharedData map is: " +
> envi.getSharedData().size() ); //try to get the counter
> ------------------------------------------------------------------
> Here it print 0, if I use evni.getSharedData().containsKey("counter"), 
> it will return false.
>
> When creating table, I call addCoprocessor() method to add Observer 
> first, then Endpoint coprocessor. I confirmed that by checking the 
> hbase log file message. I only have one region for that table during 
> the run. I confirmed by hbase shell status 'detailed' command.
>
> There is not much example I can find about how to use getSharedData(), 
> could someone help me here? What is missing in my simple code? Thanks 
> very much in advance!
>
> Thanks,
> Ming
>
Mime
View raw message