lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Fleming <afle...@hotmail.com>
Subject RE: Rebuilding index on a regular basis
Date Fri, 22 Dec 2006 05:12:33 GMT

Hi Patrik

Thanks for the thoughtful responses.  I am not a pro with Searchers yet, but it seems like
closing + opening searchers would still result in a small period of unserviceability.  I would
also like to stick to the Directory API so that I can keep the option to use FS or RAM based
indexes.  

I think a slight extension to this idea may really do the trick.  It's as follows: 

1. Create Index A + Reader A.
2. set CurrentReader = A
3. After time interval T, build Index B + Reader B. 
4. set CurrentReader = B
5. After time interval T, rebuild Index A + Reopen A
6. set CurrentReader = A

etc. 

The advantage here being that the '=' operation is atomic and indivisible - the currentReader
variable always points to a valid and up-todate index.  Although this system doesn't GUARANTEE
there won't be a service interruption, in practice if T is long enough there shouldn't be
a problem. 


Thoughts?  I'm curious if this solution reflects a misunderstanding about the way Lucene works.



Thanks, 

Adam 



----------------------------------------
> Date: Wed, 20 Dec 2006 22:11:33 -0500
> From: patrek@gmail.com
> To: java-user@lucene.apache.org
> Subject: Re: Rebuilding index on a regular basis
> 
> Hi,
> 
> How about this:
> 
> 1) You copy the files that make your index in a new folder
> 2) You update your index in that new folder (forcing if necessary, old locks
> will not be valid)
> 3) When update is completed, close your readers, and open them on the new
> index.
> 4) Copy the fresh index files to the previous location for next round, where
> you won't need the initial copy to a fresh folder.
> 
> That way, you won't have to reindex all your documents (assuming only a
> small subset needs updating) and will be able to switch to a more up to date
> index more easily and often.
> 
> Patrick
> 
> 
> On 12/20/06, Scott Sellman <ssellman@valueclick.com> wrote:
> >
> > Note: I have changed the title of this thread to match its content
> >
> > I am currently facing a similar issue.  I am dealing with a large index
> > that is constantly used and needs to be updated on a daily basis.  For
> > fear of corruption I would rather rebuild the index each time,
> > performing tests against it before using it.  However the problem I am
> > having is switching in the old index without causing service
> > interruption.  As long as queries are being made against the index I am
> > running into locking issues with the index files, preventing me from
> > putting the new index in place. Any suggestions?
> >
> > Thanks,
> > Scott
> >

_________________________________________________________________
Get the Live.com Holiday Page for recipes, gift-giving ideas, and more.
www.live.com/?addtemplate=holiday
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message