lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karel Tejnora <ka...@tejnora.cz>
Subject IndexWriter mergeSegments
Date Tue, 02 May 2006 23:29:06 GMT
Hi,
    I found a small issue when I add 10GB index to 20GB index using 
addIndexes when useCompoundFile == true.

Before compound file is created the segments info are written but points 
to non-existing coumpound file then new .tmp is created and renamed to .cfs
Between time when new segments was written and new cfs is creating Index 
reader in different application whith no care about a lock can take 
segment with empty cfs resulting in empty hits.

The lines of the code can be found in IndexWriter mergeSegments(int,int) 
btw. 696 and 712
/* segments written, new cfs has not been created */
synchronized (directory) {                 // in- & inter-process sync
        new Lock.With(directory.makeLock(COMMIT_LOCK_NAME), 
COMMIT_LOCK_TIMEOUT) {
            public Object doBody() throws IOException {
              segmentInfos.write(directory);     // commit before deleting
              deleteSegments(segmentsToDelete);  // delete now-unused 
segments
              return null;
            }
          }.run();
      }
/* create cfs */
if (useCompoundFile) {
      final Vector filesToDelete = merger.createCompoundFile(mergedName 
+ ".tmp");
      synchronized (directory) { // in- & inter-process sync
        new Lock.With(directory.makeLock(COMMIT_LOCK_NAME), 
COMMIT_LOCK_TIMEOUT) {
          public Object doBody() throws IOException {
            // make compound file visible for SegmentReaders
            directory.renameFile(mergedName + ".tmp", mergedName + ".cfs");
            // delete now unused files of segment
            deleteFiles(filesToDelete);  
            return null;
          }
        }.run();
      }
    }

I see solution in swap those two blocks. firstly create final cfs and 
than write and delete old segments.

Has anybody seen a problem with that?

I can send patch but firstly I need to find svn client in gentoo :) and 
it's to late here.
Can be smb so kind and give me link where I can find how to generate 
patch in lucene/apache way?

Thx,
Karel

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message