lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Wellnhofer <wellnho...@aevum.de>
Subject Re: [lucy-user] RE: LUCY_Folder_Open_Out_IMP at core/Lucy/Store/Folder.c line 119
Date Wed, 07 Dec 2016 11:40:03 GMT
On 06/12/2016 22:16, Gupta, Rajiv wrote:
> I thought since I'm doing read and write together I may be getting file error so I tried
to use FastUpdate method described here - http://lucy.apache.org/docs/perl/Lucy/Docs/Cookbook/FastUpdates.html

You shouldn't get any filesystem errors when searching and indexing 
simultaneously. The only error you might get under normal operation is lock 
timeouts. In this case, you should consider fast updates.

But the errors you're seeing indicate a different problem that probably won't 
be cured by fast updates.

> But now I'm more frequently getting below error.
> Error input 57 too high
> 20161206 150630 []  S_fibonacci at core/Lucy/Index/IndexManager.c line 129

This is caused by a known bug. Unfortunately, the fix wasn't committed when it 
came up for the first time:

https://lists.apache.org/thread.html/0465759f6eae2108be30c70b490d0f94ab2b5c66bfac2b32c76eb41f@1362406649@%3Cuser.lucy.apache.org%3E

I'll make sure that the fix gets into the next release.

> How many docs I should limit to commit together?

If you're (re)indexing thousands of documents and don't want searchers to be 
locked out, you should consider indexing batches of documents and sleep 
between each batch to allow concurrent searches. I'd start with several 
hundred documents per batch and and sleep for maybe 2 seconds. This thread 
contains more details:

https://lists.apache.org/thread.html/0adbe8d0b5dd6c7491a8f28008428a39f485fae58ed606475f94c636@1355773975@%3Cuser.lucy.apache.org%3E

Nick


Mime
View raw message