lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Tamm (JIRA)" <j...@apache.org>
Subject [jira] Created: (LUCENE-528) Optimization for IndexWriter.addIndexes()
Date Tue, 21 Mar 2006 22:15:34 GMT
Optimization for IndexWriter.addIndexes()
-----------------------------------------

         Key: LUCENE-528
         URL: http://issues.apache.org/jira/browse/LUCENE-528
     Project: Lucene - Java
        Type: Improvement
  Components: Index  
    Reporter: Steven Tamm
    Priority: Minor


One big performance problem with IndexWriter.addIndexes() is that it has to optimize the index
both before and after adding the segments.  When you have a very large index, to which you
are adding batches of small updates, these calls to optimize make using addIndexes() impossible.
 It makes parallel updates very frustrating.

Here is an optimized function that helps out by calling mergeSegments only on the newly added
documents.  It will try to avoid calling mergeSegments until the end, unless you're adding
a lot of documents at once.

I also have an extensive unit test that verifies that this function works correctly if people
are interested.  I gave it a different name because it has very different performance characteristics
which can make querying take longer.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message