lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <>
Subject Re: to filter or not to filter
Date Thu, 18 Aug 2005 01:11:00 GMT
On Aug 17, 2005, at 3:29 PM, Dan Funk wrote:
> Currently I'm working with a single index where content is indexed  
> by it's original printed page. I have to show the total number of  
> matching documents, so I end up running through all the hits and  
> taking an order of magnitude hit on performance as I calculate the  
> number of unique documents.  It's stupid for many many reasons.
> To correct all this, I've decided to create two (maybe three)  
> indexes for the same set of documents: in the first index there is  
> a one to one relationship between the original document and the  
> Lucene Document object.  The other index is a paragraph index,  
> where each lucene document represents a single paragraph.   I may  
> even throw in a third index where each lucene document represents a  
> logical section/chapter.
> When I'm building the search results page I'll have to execute a  
> fair number of queries. The first query will execute on the  
> Document-Index, then for each of the 10 to 2o results I'm  
> displaying at the time, I'll execute another query to find the best  
> paragraph and or section.
> Is this a reasonable solution to the problem?
> Thanks for the advice.

Just one design alternative - a Lucene index does not have to be  
homogenous in terms of the fields for a document.  So you could index  
all those various granularities into a single index with an  
additional field per document indicating whether it is a document,  
paragraph, or section/chapter.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message