lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ralf Heyde" <ralf.he...@gmx.de>
Subject AW: Lucene reorganizing indexes
Date Mon, 16 Jul 2012 20:41:55 GMT
Do you use Lucene or Solr?

We faced the problem in Solr due too big Caches, which where (re)warmed up
after a commit and the never ending full GCs.

Greets Ralf

-----Urspr√ľngliche Nachricht-----
Von: Scott Smith [mailto:ssmith@mainstreamdata.com] 
Gesendet: Montag, 16. Juli 2012 22:29
An: java-user@lucene.apache.org
Betreff: Lucene reorganizing indexes 

We have an application that has to do "real time" indexing of a number of
documents.  What it does is wake up about every 20 seconds and updates the
index with any changes that have been queued since the last time it ran.
This involves adding and deleting several hundred documents.  This is all
done in a single thread.  There can be multiple threads doing searches
simultaneous with the update thread (the searches run in a different
process).

Back in the days of 1.42, we would force an index optimization once each
day.  However, my impression is that the later versions of Lucene (we are
currently using 3.5), Lucene will often do its own reorganization based on
hitting certain criteria.  I've been told that optimizing the index is,
perhaps, no longer necessary.  Can someone describe what happens here?

The reason I'm asking about this is that we see our application periodically
using excessive amounts of kernel time (on Windows) which normally indicates
a lot of disk activity.  We are unable to align this with anything our code
is doing.  Obviously, we expect Lucene to be causing disk activity, it just
seems that the last release (we were at 3.02 before going to 3.5) severely
increased the disk activity which is interfering with other things running
on the boxes.

Does any of this make sense to anyone?  Is there an explanation?  Thoughts
about what we might do about it?

Thanks in advance.

Scott


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message