lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Mellor <pmel...@hemscott.co.uk>
Subject RE: Concurrent searching & re-indexing
Date Fri, 18 Feb 2005 09:24:15 GMT
Ok, I will change my reindex method to delete all documents and then re-add
them all, rather than using an IndexWriter to write a completely new index.

Thanks for the help on this everyone.

Paul

-----Original Message-----
From: Doug Cutting [mailto:cutting@apache.org]
Sent: 17 February 2005 22:26
To: Lucene Users List
Subject: Re: Concurrent searching & re-indexing


Paul Mellor wrote:
> I've read from various sources on the Internet that it is perfectly safe
to
> simultaneously search a Lucene index that is being updated from another
> Thread, as long as all write access to the index is synchronized.  But
does
> this apply only to updating the index (i.e. deleting and adding
documents),
> or to a complete re-indexing (i.e. create a new IndexWriter with the
> 'create' argument true and then re-add all the documents)?
[ ...]
> java.io.IOException: couldn't delete _a.f1
> at org.apache.lucene.store.FSDirectory.create(FSDirectory.java:166)
[...]
> This is running on Windows 2000.

On Windows one cannot delete a file while it is still open.  So, no, on 
Windows one cannot remove an index entirely while an IndexReader or 
Searcher is still open on it, since it is simply impossible to remove 
all the files in the index.

We might attempt to patch this by keeping a list of such files and 
attempt to delete them later (as is done when updating an index).  But 
this could cause problems, as a new index will eventually try to use 
these same file names again, and it would then conflict with the open 
IndexReader.  This is not a problem when updating an existing index, 
since filenames (except for a few which are not kept open, like 
"segments") are never reused in the lifetime of an index.  So, in order 
for such a fix to work we would need to switch to globally unique 
segment names, e.g., long random strings, rather than increasing integers.

In the meantime, the safe way to rebuild an index from scratch while 
other processes are reading it is simply to delete all of its documents, 
then start adding new ones.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


_____________________________________________________________________
This e-mail has been scanned for viruses by MCI's Internet Managed Scanning
Services - powered by MessageLabs. For further information visit
http://www.mci.com

This e-mail and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you are not the intended recipient, you should not copy, retransmit or
use the e-mail and/or files transmitted with it  and should not disclose
their contents. In such a case, please notify netwebmaster@hemscott.co.uk
and delete the message from your own system. Any opinions expressed in this
e-mail and/or files transmitted with it that do not relate to the official
business of this company are those solely of the author and should not be
interpreted as being endorsed by this company.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message