incubator-lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Karman <pe...@peknet.com>
Subject Re: [lucy-user] Managing fresh/stale indexes
Date Wed, 02 Nov 2011 02:28:50 GMT
Marvin Humphrey wrote on 11/1/11 12:36 PM:
> On Tue, Nov 01, 2011 at 11:37:30AM +0200, goran kent wrote:
>> I'm pondering about the best approach to handle fresh/stale indexes
>> while searches are happening (ie, we don't want to disrupt an active
>> search).
> 
> Once an IndexSearcher has opened successfully, it no longer needs access to
> the index dir[1] -- you can actually wipe the index dir and the IndexSearcher
> will keep functioning indefinitely.  This is achieved by caching open
> filehandles to all files the IndexSearcher could ever need access to, so that
> it will still have access even if the files are deleted[2].
>  

fwiw, this is exactly what I do.

For incremental updates, my Searcher wrapper code[1] detects when the index has
changed and re-opens itself.

For full index swap-outs, I just let existing Searchers finish and re-open
themselves when they realize the underlying index is different. When the open
filehandles eventually close, then the filesystem actually removes the old index.

[1] https://metacpan.org/module/SWISH::Prog::Lucy::Searcher

-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Mime
View raw message