hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2001) Coprocessors: Colocate arbitrary code with regions
Date Thu, 03 Dec 2009 15:57:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785368#action_12785368

Andrew Purtell commented on HBASE-2001:

.bq Would suggest that if coprocessors only need three states, then just do those. 

I'll make that clear in the documentation.

bq. First thought is that mimicing table interface might confuse more than it helps but need
to study more before I can offer stronger opinion.

There are two table interfaces currently:
1) PrivateStore -- a full table, private to the Coprocessor instance, backed by a HFile+log
2) RegionAccessor -- a table interface to the region associated with the Coprocessor, will
reject edits outside the region, otherwise allows (controlled) admin and mutation actions
on the region. Also of note here the Coprocessor hooks will be called for its own edits, just
makes things easier.

Just FYI I have the environment implemented except for PrivateStore. No classloader or ASM
bits yet; I need to study and think more about that.

Meantime the next step is hooking up Coprocessors to the HRegion via calls out to the interface
in all the right places in HRegion, and possibly a few up in HRegionServer. 

> Coprocessors: Colocate arbitrary code with regions
> --------------------------------------------------
>                 Key: HBASE-2001
>                 URL: https://issues.apache.org/jira/browse/HBASE-2001
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>         Attachments: asm-3.2-bin.zip, asm-transformations.pdf, org.apache.hadoop.hbase.HCoprocessor.java,
> "Support arbitrary code that runs run next to each region in table. As regions split
and move, coprocessor code should automatically  move also."
> Use classloader which looks on HDFS.
> Associate a list of classes to load with each table. Put this in HRI so it inherits from
table but can be changed on a per region basis (so then those region specific changes can
inherited by daughters). 
> Not completely arbitrary code, should require implementation of an interface with callbacks
> * Open
> * Close
> * Split
> * Compact
> * (Multi)get and scanner next()
> * (Multi)put
> * (Multi)delete
> Add method to HRegionInterface for invoking coprocessor methods and retrieving results.
> Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions
for coprocessor methods and consistent/controlled access to internals: store access, threading,
persistent and ephemeral state, scratch storage, etc. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message