hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean x66079" <dean.hil...@broadridge.com>
Subject indexing libraries lucene/katta/solr all have on major issue with hbase????
Date Sun, 19 Jun 2011 18:57:34 GMT
First, this is easier to explain with context of hibernate
If you run a query select * from xxx where yyy and page the results using JPA query.setMaxResults(100)
and query.setFirstResult, then paging the results gets slower over time which is fine for
webpages maybe where the user is clicking next page, etc. etc.

If you run the same query using ScrollableResultSet, then paging does not get slower at all
since the database maintains a cursor as to where it left off until the ScrollableResultSet
is closed so it maintains it's speed.

I am trying to use Lucene right now(to be replaced with SOLR/Katta later) to create indexes
but we need to process over a large result set paging each 100 and we really don't want to
"start over".  We want the performance of something like ScrollableResultSet.

Does anyone know of any indexing libraries that do it more like ScrollableResultSet instead
of getting slower over time?

Thanks,
Dean

This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message