lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen" <jason.rutherg...@gmail.com>
Subject Re: SegmentReader with custom setting of deletedDocs, single reusable FieldsReader
Date Mon, 30 Jun 2008 02:41:51 GMT
To complete the removal any low level synchronization issues, seems like a
good idea to add a clone method to CSIndexInput that clones the base
IndexInput to remove the synchronization in readInternal.

On Sun, Jun 29, 2008 at 1:44 PM, Jason Rutherglen <
jason.rutherglen@gmail.com> wrote:

> I voted for http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734about pread not
working on Windows.  After thinking more about the pool of
> RandomAccessFiles I think LUCENE-753<https://issues.apache.org/jira/browse/LUCENE-753>is
the best solution.  I am not sure how much work nor if pool of
> RandomAccessFiles creates more synchronization problems and if it is only to
> benefit windows, does not seem worthwhile.
>
> Seems like LUCENE-753 needs the conditional part on the OS and it can be
> committed.
>
> On Sun, Jun 29, 2008 at 10:47 AM, Yonik Seeley <yonik@apache.org> wrote:
>
>> On Sun, Jun 29, 2008 at 9:42 AM, Jason Rutherglen
>> <jason.rutherglen@gmail.com> wrote:
>> > IndexReader.document as it is is really a lame duck.  The
>> > IndexReader.document call being synchronized at the top level drags down
>> the
>> > performance of systems that store data in Lucene.  A single file
>> descriptor
>> > for all threads on an index that is constantly returning results with
>> fields
>> > is a serious problem.  Users are always complaining about this issue and
>> now
>> > I know why.
>>
>> Each part of the index (e.g. tis, frq) is actually only covered by a
>> single file descriptor by default - stored fields aren't unique in
>> that regard.
>>
>> It's probably the case that the stored fields of a given document are
>> much less likely to be in OS cache though... and in that case having
>> multiple requests in-flight to the disk could improve things.
>>
>> On anything except Windows, using pread may be the answer (after the
>> other synchronization is also removed of course):
>> https://issues.apache.org/jira/browse/LUCENE-753
>>
>> -Yonik
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>

Mime
View raw message