lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Karman <pe...@peknet.com>
Subject Re: [lucy-user] C library, how to check index is healthy
Date Tue, 14 Feb 2017 15:15:30 GMT
Nick Wellnhofer wrote on 2/14/17 9:03 AM:
> On 13/02/2017 20:44, Serkan Mulayim wrote:
>> 1- How do we check that the index is healthy for SEARCHING (e.g. creating a
>> searcher) without a crash? As I see there is no problem in creating a Searcher
>> even if there is a lock (write.lock or merge.lock)
>
> First of all, Lucy should never "crash" in the sense of a segfault. If it does,
> this is a bug that should be reported.
>
> Unless your index is on a shared volume like NFS, it can always be searched.
>

One trick to keep in mind is that if the index underlying a Searcher changes (as 
through indexing or document deletion), you must detect that change and open a 
new Searcher. Because of mmap it's very fast to spawn a new Searcher, but 
sometimes you'll see stale results if you persist one too long.

An example of how Dezi does that here:
https://metacpan.org/source/KARMAN/Dezi-App-0.014/lib/Dezi/Lucy/Searcher.pm#L406

tl;dr is that Dezi writes its own index metadata header that includes a UUID and 
timestamp for the last time the index was updated, and checks that UUID against 
the current Searcher to know if it is stale and needs to be re-created.


-- 
Peter Karman  .  https://peknet.com/  .  https://keybase.io/peterkarman

Mime
View raw message