lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lock-less commits
Date Sun, 20 Aug 2006 18:35:31 GMT
Yonik Seeley wrote:
> On 8/20/06, Michael McCandless <lucene@mikemccandless.com> wrote:
>> On deletable: yes, I'm currently GC'ing unused segments by doing a
>> full directory listing.
> 
> Actually, you could get a full directory listing once per IndexWriter
> and keep the results up-to-date in memory (including deletes that
> fail).  No need for a "deletable" file, and the directory-listing hit
> is only taken once per IndexWriter instance, not once per merge.

Excellent!  I will take this approach.

> IndexWriters also need to open IndexReaders (SegmentReaders) for
> merging... I don't know if you needed to modify SegmentReader in a way
> that reduces performance, but if so it might be possible to make a
> special package protected factory method for use by IndexWriter that
> regains any performance loss by making certain assumptions.

So far, I believe my mods to SegmentReader should not affect
performance.  It's just when instantiating the SegmentInfos (well,
SegmentInfos.read()) that I do a directory listing to find the latest
"generation" segments.N file.  When IndexWriter creates the
SegmentMerger, since it uses its own SegmentInfo's to get() each
SegmentReader, all the necessary details (which .del.N and norms files
are "current") are in the SegmentInfo and so SegmentReader doesn't
need to do any extra "work".  Still this is a good suggestion to
remember for future work.

Thanks for all the feedback!

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message