lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Gardner (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
Date Wed, 16 Jul 2008 19:33:31 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614074#action_12614074
] 

Brian Gardner commented on LUCENE-753:
--------------------------------------

This probably doesn't help much, but I implemented a pool and submitted a patch very similar
to the SeparateFile approach.  Before being directed to this thread: 
https://issues.apache.org/jira/browse/LUCENE-1337

In our implementation the synchronization/lack of concurrency has been a big issue for us.
 On several occasions we've had to remove new features that perform searches from frequently
hit pages, because threads build up waiting for synchronized access to the underlying files.
 It is possible that I would still have issue even with my patch, considering from my tests
that I'm only increasing throughput by 300%,  but it would be easier for me to tune and scale
my application since resource utilization and contention would be visible from the OS level.



> At first blush, sounds a bit too complex for the benefits.

My vote is that the benefits outway the complexity, especially considering it's an out-of-the
box solutions that works well for all platforms and single threaded as well as multi-threaded
envirnments.  If it's helpful, I can spend the time to implement some of the missing feature(s)
of the pool that will be needed for it to be an acceptable solution (i.e, shared access once
a file has been deleted, and perhaps a time-based closing mechanism).

> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
>                 Key: LUCENE-753
>                 URL: https://issues.apache.org/jira/browse/LUCENE-753
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>            Reporter: Yonik Seeley
>         Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
FileReadTest.java, FileReadTest.java, FileReadTest.java, FSIndexInput.patch, FSIndexInput.patch,
lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the underlying
file.
> This could mitigate any MT performance drop caused by reducing the number of files in
the index format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message