lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitaly Funstein <vfunst...@gmail.com>
Subject NRT index readers and new commits
Date Thu, 30 Jan 2014 08:03:55 GMT
Suppose I have an IndexReader instance obtained with this API:

DirectoryReader.open(IndexWriter, boolean);

(I actually use a ReaderManager in front of it, but that's beside the
point).

There is no manual commit happening prior to this call. Now, I would like
to keep this reader around until no longer needed, i.e. until the app is
done with the data this reader will see. Since the index is live, there may
be new data added after the reader is returned, of course - followed by one
or more commits at arbitrary points in time.

The question is - what will happen to the flushed segment files this reader
is backed by, when there is a later commit on the writer that is tied to
the NRT reader? If I'm reading the code correctly, IndexFileDeleter will
detect reference counts to those old segment files reaching 0 and try to
delete them. This is because IndexWriter.getReader(boolean) doesn't
checkpoint with the IndexFileDeleter associated with it, which would
increase ref counts on the managed files. Also, actually closing this
reader appears to have provide no feedback to the backing writer, it only
closes the data stream(s) but doesn't seem to release used segment files to
the deleter...

Does all this just rely on that most OSs will allow deletion of a file that
is still opened (Windows being a notable exception)? It seems the whole
IndexWriter.deletePendingFiles() API exists to work around the situation
when it's not allowed... And is it valid to assume it's a safe thing to do,
even when the OS supports it?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message