lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: [jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
Date Wed, 20 Dec 2006 03:04:53 GMT
On 12/19/06, robert engels <rengels@ix.netcom.com> wrote:
> That is what I did in my implementation.
>
> It is easy to implement this as a protected abstract method in
> IndexInput, which uses synchronization (and seek(), read(byte
> [],off,len). In the 'real" version just override it to do it properly
> without using seek().

You can synchronize and protect other calls to that same pread method,
but it breaks if it's called concurrently with most other methods
because they aren't synchronized.  It's not "thread-safe" in the
normal sense of the word.

One would have to specify that if you were going to use an IndexInput
from multiple threads at once, that you could *only* use the pread
method, and pretty much no others without external synchronization.
That's doable I guess, but complicated to try to explain to others.

-Yonik


>
> On Dec 19, 2006, at 7:40 PM, Yonik Seeley (JIRA) wrote:
>
> >     [ http://issues.apache.org/jira/browse/LUCENE-753?
> > page=comments#action_12459805 ]
> >
> > Yonik Seeley commented on LUCENE-753:
> > -------------------------------------
> >
> > CSIndexInput synchronization could also be elimitated if there was
> > a pread added to IndexInput
> >
> >   public abstract void readBytes(byte[] b, int offset, int len,
> > long fileposition)
> >
> > Unfortunately, that would break any custom Directory based
> > implementations out there, and we can't provide a suitable default
> > with seek & read because we don't know what object to synchronize on.
> > Worth it or not???
> >
> >> Use NIO positional read to avoid synchronization in FSIndexInput
> >> ----------------------------------------------------------------
> >>
> >>                 Key: LUCENE-753
> >>                 URL: http://issues.apache.org/jira/browse/LUCENE-753
> >>             Project: Lucene - Java
> >>          Issue Type: New Feature
> >>          Components: Store
> >>            Reporter: Yonik Seeley
> >>         Attachments: FSIndexInput.patch
> >>
> >>
> >> As suggested by Doug, we could use NIO pread to avoid
> >> synchronization on the underlying file.
> >> This could mitigate any MT performance drop caused by reducing the
> >> number of files in the index format.
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message