lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Mastracci (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
Date Fri, 08 Aug 2008 23:06:44 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621091#action_12621091
] 

Matthew Mastracci commented on LUCENE-753:
------------------------------------------

bq. Is the index itself corrupt, ie, NIOFSDirectory did something bad when writing the index?
Or, is it only in reading the index with NIOFSDirectory that you see this? IE, can you swap
in FSDirectory on your existing index and the problem goes away?

I haven't seen any issues with writing the index under NIOFSDirectory.  The failures seem
to happen only when reading.  When I switch to FSDirectory (or MMapDirectory), the same index
that fails under NIOFSDirectory works flawlessly (indicating that the index is not corrupt).

The error with NIOFSDirectory is determinate and repeatable (same error every time, same location,
same query during warmup).

I couldn't reproduce this on a smaller index, unfortunately.


> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
>                 Key: LUCENE-753
>                 URL: https://issues.apache.org/jira/browse/LUCENE-753
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>            Reporter: Yonik Seeley
>         Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
FileReadTest.java, FileReadTest.java, FileReadTest.java, FSDirectoryPool.patch, FSIndexInput.patch,
FSIndexInput.patch, lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the underlying
file.
> This could mitigate any MT performance drop caused by reducing the number of files in
the index format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message