hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mingjie Lai <mjla...@gmail.com>
Subject Re: Loaded Coprocessor
Date Tue, 08 Feb 2011 20:12:14 GMT

Hi JS.

Here are some hints to help you debug:
- make sure you have the jar copied to all the region servers if you 
want it to sit on local fs, or you can put it to hdfs so all RS can 
access it without copying arount.
- remember to disable, and enable the table to make sure the attribute 
be loaded
- if you still cannot see it working, please post your RS log after the 
time of table enabled.

Loading cp classes from file system lacks of testing, so there might be 
some issues. I added a jira:
https://issues.apache.org/jira/browse/HBASE-3516 to add test cases back 
for class loading.

Another workaround is to use configuration to test your coprocessor.

This has been tested a lot among us. The only problem is that it would 
be applied to all your tables. You can use this method if there is an 
issue for cp class loading from jar files.

Thanks,
Mingjie


On 02/08/2011 10:01 AM, js_84@email.de wrote:
> Hi,
> thanks for your feedback!
>
> Just a follow up question;
> According to the TableDescriptor
> {NAME =>  'test3', Coprocessor$2 =>  '/Users/hbase/CP.jar:Coprocessor.PreGet:12',
Coprocessor$1 =>  '/Users/hbase/CP.jar:Coprocessor.PrePut:10',
> FAMILIES =>  [{NAME =>  'cf', BLOOMFILTER =>  'NONE', REPLICATION_SCOPE =>
 '0', COMPRESSION =>  'NONE', VERSIONS =>  '3', TTL =>  '2147483647', BLOCKSIZE =>
 '65536', IN_MEMORY =>  'false', BLOCKCACHE =>  'true'}]}
>
> the coprocessors is specified, but actually neither of them is executed when executing
a put; So I am not sure whether they are actually loaded (what happens if if jar file is not
found? ->  this should trigger at least a warning).
>
> Any ideas/pointers why this fails (HBAse Version 0.91.0-SNAPSHOT, r1065918)?
>
> package Coprocessor;
> public class PrePut extends BaseRegionObserverCoprocessor {
>   @Override
>   public void prePut( RegionCoprocessorEnvironment e,  Map<byte[],
>            List<KeyValue>>  familyMap, boolean writeToWAL) throws IOException
{
>
>           //bypass normal system call
>           e.bypass();
>      }
> }
>
>
> Thanks again!
> JS
>
>
> -----Urspr√ľngliche Nachricht-----
> Von: "Andrew Purtell"<apurtell@apache.org>
> Gesendet: 07.02.2011 23:42:35
> An: dev@hbase.apache.org
> Betreff: Re: Loaded Coprocessor
>
>> is there any way to check which Coprocessors Classes are
>> currently loaded? Preferebly using the shell...
>
> Not yet. Good idea, filed: https://issues.apache.org/jira/browse/HBASE-3512
>
>> And related: Can one dynamically load Coprocessors for a
>> running hbase system?
>
> Currently coprocessors are loaded statically via configuration or dynamically upon table
open if there are any table attributes that specify it (attribute name begins with "Coprocessor").
IIRC the javadoc on the coprocessors package contains detail on this. In short:
>
>      HTableDescriptor htd;
>      // ...
>      htd.setValue("Coprocessor$foo", "hdfs://namenode:port/path/to/coprocessor.jar");
>
> A table descriptor can be updated if such specifications are not made a table create
time. See javadoc for HBaseAdmin.
>
> However there is currently no mechanism for triggering additional loads on tables already
open or loading at arbitrary times. There hasn't been a clearly articulated need for this
yet.
>
> Best regards,
>
>      - Andy
>
> Problems worthy of attack prove their worth by hitting back.
>    - Piet Hein (via Tom White)
>
>
> --- On Mon, 2/7/11, js_84@email.de<js_84@email.de>  wrote:
>
>> From: js_84@email.de<js_84@email.de>
>> Subject: Loaded Coprocessor
>> To: dev@hbase.apache.org
>> Date: Monday, February 7, 2011, 1:48 AM
>> Hi,
>> is there any way to check which Coprocessors Classes are
>> currently loaded? Preferebly using the shell...
>> And related: Can one dynamically load Coprocessors for a
>> running hbase system?
>>
>> Thanks for your support!
>> JS
>>
>

Mime
View raw message