hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2001) Coprocessors: Colocate arbitrary code with regions
Date Mon, 30 Nov 2009 21:07:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783841#action_12783841
] 

stack commented on HBASE-2001:
------------------------------

Comments on doc:

"Regions contain references to the coprocessor implementation classes associated with them."

Q: On above, its indeed the classes, not objects?  Objects can cross the split?  Not easily
anyways.

Do we need both closing and pendingClose?  I'd think just one?  (Change pendingOpen to opening
and keep closing ditching pendingClose).  I don't know for sure.  Just asking.

Why no control over flush?  Maybe it would want to hold up a flush?  You think that too dangerous?

On the interface:

Drop the "H" from Coprocessor class I'd say.  Thats the new fashion these days.

It looks great.

Rather, should we do the java Events model where one method gets all event types, the passed
in object says that the event is.  In the method, first thing you check if its an event you
are interested in?  Makes things easier to implement especially if you are only implementing
part of the functionality.  This model may not make sense though for this context or may be
overkill (See java.util.EventObject and some of its implementations).

Will Coprocessors make for lots of new object instantiations?  Its going to be invoked on
each Get and Scan.

The logging interface seems odd.  Why have new define?  Why not just use apache logging?

Should we be extracting an Interface from Region so we can have a Region implemetention and
so your Coprocessor can have an implementation too?  We sort of did something like with the
"Incommon" interface we have for testing that has allows for implementations that run the
same tests only now against the Region and then against the client-side.   Extracting a 'official'
Region interface sounds grand to me... would help with testing?

How does the PrivateStore persist?  Where?  What you thinking?

Looks great.

> Coprocessors: Colocate arbitrary code with regions
> --------------------------------------------------
>
>                 Key: HBASE-2001
>                 URL: https://issues.apache.org/jira/browse/HBASE-2001
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>         Attachments: asm-3.2-bin.zip, asm-transformations.pdf, org.apache.hadoop.hbase.HCoprocessor.java,
org.apache.hadoop.hbase.HCoprocessor.pdf
>
>
> "Support arbitrary code that runs run next to each region in table. As regions split
and move, coprocessor code should automatically  move also."
> Use classloader which looks on HDFS.
> Associate a list of classes to load with each table. Put this in HRI so it inherits from
table but can be changed on a per region basis (so then those region specific changes can
inherited by daughters). 
> Not completely arbitrary code, should require implementation of an interface with callbacks
for:
> * Open
> * Close
> * Split
> * Compact
> * (Multi)get and scanner next()
> * (Multi)put
> * (Multi)delete
> Add method to HRegionInterface for invoking coprocessor methods and retrieving results.
 
> Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions
for coprocessor methods and consistent/controlled access to internals: store access, threading,
persistent and ephemeral state, scratch storage, etc. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message