hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-2001) Coprocessors: Colocate user code with regions
Date Sun, 14 Feb 2010 00:12:28 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Purtell updated HBASE-2001:

    Attachment: HBASE-2001.patch.gz

Latest patch has working simple unit tests for MapReduce.

There is also a working unit test for on-demand loading of coprocessor classes from jars referenced
via table attributes:

Tables can have any number of attributes with keys that start with "Coprocessor" and values
of the form <path>:<class>:<priority>, e.g.

{{'Coprocessor$1' => 'hdfs://localhost:8020/hbase/coprocessors/test.jar:Test:1000'}}
{{'Coprocessor$2' => '/hbase/coprocessors/test2.jar:AnotherTest:1001'}}

{{<path>}} must point to a jar, can be on any filesystem supported by the Hadoop FileSystem

{{<class>}} is the coprocessor implementation class. A jar can contain more than one
coprocessor implementation, but only one can be specified at a time in each table attribute.

{{<priority>}} is an integer. Coprocessors are executed in order according to the natural
ordering of the int. Coprocessors can optionally abort actions. So typically one would want
to put authoritative CPs (security policy implementations, perhaps) ahead of observers. 

> Coprocessors: Colocate user code with regions
> ---------------------------------------------
>                 Key: HBASE-2001
>                 URL: https://issues.apache.org/jira/browse/HBASE-2001
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>         Attachments: asm-3.2-bin.zip, asm-transformations.pdf, HBASE-2001.patch.gz
> Support user code that runs run next to each region in table. As regions split and move,
coprocessor code should automatically  move also.
> Use classloader which looks on HDFS.
> Associate a list of classes to load with each table. Put this in HRI so it inherits from
table but can be changed on a per region basis (so then those region specific changes can
inherited by daughters). 
> Not completely arbitrary code, should require implementation of an interface with callbacks
> * Open
> * Close
> * Split
> * Compact
> * (Multi)get and scanner next()
> * (Multi)put
> * (Multi)delete
> Add method to HRegionInterface for invoking coprocessor methods and retrieving results.
> Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions
for coprocessor methods and consistent/controlled access to internals: store access, threading,
persistent and ephemeral state, scratch storage, etc. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message