lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vsevel <v.se...@lombardodier.com>
Subject Searching while optimizing
Date Mon, 23 Nov 2009 20:20:31 GMT

Hi, I am using lucene 2.9.1 to index a continuous flow of events. My server
keeps an index writer open at all time and write events as groups of a few
hundred followed by a commit. While writing, users invoke my server to
perform searches. Once a day I optimize the index, while writes happens and
searches may happen. I adopted the following strategy:

for every search I open a new IndexSearcher of the reader of the writer. I
execute the search, fetch the documents and finally close the searcher.
Specifically, I never close the reader, nor the writer.

Q: is that a reasonnable strategy?

I found out that my 40Gb index grew up to 200Gb while the number of docs
stayed put at 30 millions. I am suspecting that a search during the optimize
caused this situation, as described in the index writer javadoc (about
refreshing readers during an optimize).

Q: is that the likely cause? is getting a reader of the writer just as "bad"
as refreshing a reader during an optimize? how can I avoid this behavior?
should I just deny searches while optimizing?

question on the side: is there any way to interrupt a search that takes too
long? for instance by setting a boolean from another thread on the searcher
currently performing the search.

thanks,
vincent
-- 
View this message in context: http://old.nabble.com/Searching-while-optimizing-tp26485138p26485138.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message