hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: why not introduce reverse index into hbase for multi-dimensional query
Date Fri, 18 Jul 2014 07:40:55 GMT
See:
http://www.infoq.com/articles/LuceneHbase

Note: We did this early in 2011 but couldn’t talk about it for a while. 

SOLR / Elastic Search would follow the same pattern. 

Note that depending on what you’re indexing, the size of the index(s)  could be larger than
the base table by a couple of orders in magnitude. 

If you wanted to tie SOLR to HBase for an in memory index you have a decision to make. 
Do you update the index data in hbase and have an eventual consistency model where it will
take some time x (variable and measured in minutes to hours) before the data is available
to the index, or do you want to update the data in memory and then persist to hbase. 

We built the index and updated the index in HBase because we didn’t care about the eventual
consistency. So we had to modify the flow of information. 

If you are writing to SOLR directly, then SOLR has to persist in to HBase, and then you will
have to deal with the issue of if SOLR isn’t available what do you do with the data? 
(Assume that you could on error write to HBase.) 

Its definitely an option but you would also have to write the co-processor code to handle
the index writes as you update the base table. 

HTH

-Mike

On Jul 16, 2014, at 5:51 AM, 张景鹏 <zhangjingpeng@jd.com> wrote:

> Sorry ,yes it's Solr
> When I combin hbase with solr ,I find it have a good performance . I store the colums
which should be query as condition and hbase rowkey in solr and all colums stored in hbase
. so I think hbase can add the solr's(lucene) some index function as hbase two secondary index.
> I recommend you to look at this article which will have a help.  
> http://static.druid.io/docs/druid.pdf 
> 
> 
> 
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com] 
> 发送时间: 2014年7月16日 10:58
> 收件人: user@hbase.apache.org
> 主题: Re: why not introduce reverse index into hbase for multi-dimensional query
> 
> Can you describe reverse index in bit more detail ?
> 
> BTW sorl should be solr, right ?
> 
> 
> On Tue, Jul 15, 2014 at 7:42 PM, 张景鹏 <zhangjingpeng@jd.com> wrote:
> 
>> why not introduce reverse index into hbase for multi-dimensional query ?
>> looks like sorl + hbase
>> 


Mime
View raw message