lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doron Cohen <DOR...@il.ibm.com>
Subject Re: IndexReader.isCurrent in presence of many files
Date Sat, 12 May 2007 02:18:39 GMT
Chris Hostetter <hossman_lucene@fucit.org> wrote on 11/05/2007 17:10:54:

>
> : If this really turns to be related to having many files in the index
> : dir, could we maintain SEGMENTS_N files in a sub-directory..?
>
> I haven't done much experimenting / performance testing of File
> operations in Java, but just from looking at the java1.4.2 javadocs it
> seems like it *might* be possible to make the FindSegmentsFile class
> faster if line 504 (of SegmentInfos.java r518529) used a FilenameFilter.
>
> I say might because:
>   1) i have no idea how Java uses the filter udner the covers when
dealing
>      with the filesystem .. worst case it would help keep the array size
>      down.

That's right - AFAIK Java's File.list(filter) just iterates all the files
in the filesystem directory and adds those accepted by the filter to
its result. So this is not supposed to help.

>   2) it's not immediately clear to me if that code path is ever used ...
i
>      suspect most clients are using the lucene Directory constructor and
>      not the java.io.File constructor ... Directory.list does not support
>      FilenameFilters. so we can't try the same thing on that code path
>      with making some other changes.

I'm aware that anything like this would call for an API addition to
o.a.l.s.Directory.

Also, Yonik has a point about backwards compatibility
vs. speed on NFS, though I don't know if this only shows on NFS.

However, isCurrent() may be called before every query.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message