lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: Lock-less commits
Date Fri, 18 Aug 2006 17:24:32 GMT
On 8/18/06, Michael McCandless <lucene@mikemccandless.com> wrote:
> It could in theory lead to starvation but this should be rare in
> practice unless you have an IndexWriter that's constantly committing.

An index with a small mergeFactor (say 2) and a small maxBufferedDocs
(default 10), would have segments deleted every
mergeFactor*maxBufferedDocs when rapidly adding documents.  It might
help to start opening segments with the *last* segment, where segment
deletions are most likely to happen.

Also, when loading a .del file, how would one tell if it didn't exist
or if it was just deleted?
I guess one would always need to write a .del file even if no docs
were deleted.  Or, one could just order the deletes (delete optional
files in a segment last).

One would also have to worry about partially deleted segments on
Windows... while removing a segment, some of the files might fail to
delete (due to still being open) and some might succeed.

> Well, in my current implementation I don't have a truly asynchronous
> deleter.

Yeah, that's not really needed I guess.

This idea is worth kicking around more for the future (maybe for when
the index format changes again), but it's probably too much change for
right now (Lucene 2.0.x), right?

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message