hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene Koontz <ekoo...@hiro-tan.org>
Subject Re: what's the roadmap of secondary index of hbase?
Date Fri, 25 Feb 2011 21:47:05 GMT
On 2/25/11 12:43 PM, Jonathan Gray wrote:
> Cool.  Plans for a design phase that we can collaborate on?
Hi Jon,

I'm thinking that we could use a coprocessor that watches the 
Write-Ahead Log (using the WAL-edit operations  
https://issues.apache.org/jira/browse/HBASE-3257 "Coprocessors: Extend 
server side integration API to include HLog operations"). This 
coprocessor would write these edits, perhaps filtering or transforming 
them, and enqueing the results in a global queue. A separate process 
would be responsible for pulling operations off the queue and using 
HBase client operations to do the insert into a secondary index table 
appropriate for that operation.
     Perhaps we could use some of the work that the Lily people have 
done with HBase indexing (see 
http://www.lilyproject.org/lily/about/playground/hbaseindexes.html) in 
order to do the edit->hbase operation transformations and the secondary 
index table creation.

View raw message