lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antony Bowesman <>
Subject Re: Using Lucene partly as DB and 'joining' search results.
Date Tue, 15 Apr 2008 05:23:37 GMT
Chris Hostetter wrote:
> you can't ... that's why i said you'd need to rebuild the smaller index 
> completley on a periodic basis (going in the same order as the docs in the 

Mmm, the annotations would only be stored in the index.  It would be possible to 
store them elsewhere, so I can investigate that, in which case the rebuild would 
be possible.

> i can also imagine a situation where you break both indexes up into lots 
> of pieces (shards) and use a MultiReader over lots of ParallelReaders ... 
> that way you have much smaller "small" indexes to rebuild when someone 
> annotates an email -- and if hte shards are organized by date, you're less 
> likely to ever need to rebuild many of them since people will tend to 

Data will be 'sharded' anyway, by date of some granularity.  Looking at the 
source for MultiReader/MultiSearcher, they are single threaded.  Is there a 
performance trade off between single-thread/many small indexes and 
single-thread/some large indexes.  Can a MultiReader work with one..n reader per 
thread, something like a thread pool of IndexReaders.  I expect it would be 
faster to run the searches in parallel?

> Disclaimer: all of this is purely brainstorming, i've never actually tried 
> anything like this, it may be more trouble then it's worth.

:) Thanks for the sounding board - it's always useful to get new ideas!

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message