hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Helmling (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2038) Coprocessors: Region level indexing
Date Sun, 21 Nov 2010 20:01:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12934341#action_12934341

Gary Helmling commented on HBASE-2038:

Hi Alex,

I'm not familiar with the internal IHBase code, but I'll provide whatever info I can on coprocessors.

bq. 1) Are coprocessors meant to be stateless? If not, then I assume that one instance is
created and "assigned" to a region and that CP implementation should be thread-safe (e.g.
multiple scanners can be handled at the same time for the regions).

No, coprocessor implementations do not need to be stateless.  If anything you'll need state
for many interesting applications.  A single coprocessor instance is created per configured
coprocessor implementation on region load.  You can treat the postOpen() and preClose() methods
as init() and destroy() methods in your implementation.  And yes, coprocessor implementations
need to be thread safe.

bq. 2) During batch scan (smth which was added in trunk but wasn't supported in previous HBase
versions, and hence current IHBase implementation doesn't take it into account) we need to
return multiple rows from scan's next() method. It looks like if we apply current approach
(from current IHBase implementation) of "fast forwarding" to next value we'll only fastforward
scan to the first value of those to return. 

Sorry, I'm not familiar with how IHBase handles this or what changed in the scanner API, but
I'm guessing RegionObserver.preScannerNext() does provide much help in this fast-forwarding
use case.  It seems like this would need much deeper hooks into HRegion.RegionScanner to interact
with the positioning code.  Alternately, you could expose your own "indexed scanner" functionality
via the dynamic rpc hooks (HTable.coprocessorExec()), but that would require the client to
differentiate on indexed vs. non-indexed usage and doesn't provide the transparency you're
looking for.

bq. 3) Is it in general a good idea to take this initiave (transform IHBase implementation
to CP-based one) by me?

Sorry again, I don't have much of an answer on this one.  I'll help on anything I can on the
coprocessor side of things, though!

> Coprocessors: Region level indexing
> -----------------------------------
>                 Key: HBASE-2038
>                 URL: https://issues.apache.org/jira/browse/HBASE-2038
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Andrew Purtell
>            Priority: Minor
> HBASE-2037 is a good candidate to be done as coprocessor. It also serve as a good goalpost
for coprocessor environment design -- there should be enough of it so region level indexing
can be reimplemented as a coprocessor without any loss of functionality. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message