lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie <ja...@stimulussoft.com>
Subject combine results from multiple queries & sort
Date Wed, 14 Mar 2012 08:01:35 GMT
Greetings!

First off, I realize Lucene is a search engine and therefore does not 
possess many of the features of a database. That being said, I have 
encountered a particular use case where I need to lookup potentially 
thousands of records in a Lucene index based upon an ID (a String field 
in the index). This data also needs be sorted based upon any chosen 
field in the index. In pseudo code, this is how its currently done:

String[] ids = { "123aeeff", "34eacc", ...}

results.clear();
StringBuffer lookupQuery = new StringBuffer()
for (int i=0; i<ids.size();i++) {
        lookupQuery.append(ids.get(i))
        lookupQuery.append(" ")
         if ((i+1) % 1024 == 0) {
             search(lookupQuery.toString())
              lookupQuery = new StringBuffer()
        }
  }
if (lookupQuery.length()>0) {
         search(lookupQuery.toString())
}

As you can see, in a loop, Lucene queries are constructed into a maximum 
of 1024 terms, for example, consisting of IDS "123aeeff 34eacc ..". 
After each query in the loop is constructed, a search is executed and 
then the results are combined into a single linkedlist (this is done in 
the search function). This works well aside from two outstanding questions:

 1. Is executing separate search queries, the best way to lookup
    thousands of records in an index? Is there a more efficient way to
    lookup thousands of records based upon ID?
 2. The results are unsorted after they are combined into a single
    linkedlist. What is the best way to sort the combined results based
    upon any chosen field in the lucene index? Is there a way to do that
    would leverage Lucene's inbuilt sort abilities?

Many thanks for your consideration

Jamie


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message