lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen" <jason.rutherg...@gmail.com>
Subject Synchronization bottlenecks
Date Thu, 12 Jun 2008 15:00:15 GMT
I have seen this discussed before but with no conclusion.  It is safe to say
that SegmentReader.isDeleted is a synchronization bottleneck.  When using a
single IndexReader per query for highly concurrent application such as a web
application, with the index entirely in the system cache, the maximum the
CPU goes to is 75%-80% on a dual 2 core server.  I have not tested on a dual
quad core but would expect more significant performance degradation.  What
is the use case for SegmentReader.isDeleted being synchronized?  Most Lucene
applications are batch oriented and so performing in memory updates that
need to be reflected immediately in a search is incongruent.  I am sure this
is supposed to be taken care of with the deprecation of
IndexReader.deleteDocument.  I do not agree with deprecating
IndexReader.deleteDocument as IndexWriter does not offer a way to work
directly at the doc level.  If the system has a mix of DirectoryIndexReaders
and non-DirectoryIndexReaders (such as InstantiatedIndexReader) then
IndexWriter will not work.

Are there any other unnecessary synchronization points related to
IndexReader?

Mime
View raw message