lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Pinkerton (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
Date Mon, 30 Jun 2008 23:30:46 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609401#action_12609401
] 

Brian Pinkerton commented on LUCENE-753:
----------------------------------------

I was curious about the discrepancy between the ChannelPread implementation and the SeparateFile
implementation that Yonik saw.  At least on Mac OS X, the kernel implementation of read is
virtually the same as pread, so there shouldn't be any appreciable performance difference
unless the VM is doing something funny.  Sure enough, the implementations of read() under
RandomAccessFile and read() under FileChannel are totally different.  The former relies on
either a buffer allocated either on the stack or by malloc, while the latter allocates a native
buffer and copies the results to the original array.

Switching to a native buffer in the benchmark yields identical results for ChannelPread and
SeparateFile on 1.5 and 1.6 on OS X.  I'm attaching an implementation of ChannelPreadDirect
that uses a native buffer.

This may be a moot point because any implementation inside Lucene needs to consume a byte[]
and not a ByteBuffer, but at least it's informative.



> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
>                 Key: LUCENE-753
>                 URL: https://issues.apache.org/jira/browse/LUCENE-753
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>            Reporter: Yonik Seeley
>         Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
FileReadTest.java, FileReadTest.java, FSIndexInput.patch, FSIndexInput.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the underlying
file.
> This could mitigate any MT performance drop caused by reducing the number of files in
the index format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message