lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2056) Should NIOFSDir use direct ByteBuffers?
Date Tue, 15 Jun 2010 10:23:23 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878921#action_12878921
] 

Michael McCandless commented on LUCENE-2056:
--------------------------------------------

It's remotely possible that using direct byte buffers (in the above patch) works around the
nasty Sun JVM bug (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734) that makes
NIOFSDirectory useless on windows...

Can someone w/ access to a multi-CPU/core Windows box test this?

You just need an existing index, and then something like this alg (searches w/ 4 threads)
EXCEPT you have to temporarily edit FSDirectory.java to return this DirectNIOFSDirectory on
Windows:

{noformat}

analyzer=org.apache.lucene.analysis.core.WhitespaceAnalyzer
directory=FSDirectory
work.dir = /x/lucene/trunkwiki

log.step=100000

search.num.hits=10

query.maker=org.apache.lucene.benchmark.byTask.feeds.FileBasedQueryMaker
file.query.maker.file = queries.txt

# task at this depth or less would print when they start
task.max.depth.log=2

log.queries=true
# -------------------------------------------------------------------------------------

{ "Rounds"

    OpenReader
    [ { "topDocs" Search > : 6.0s }: 4
    CloseReader

    RepSumByPref topDocs

    NewRound

} : 10
{noformat}

> Should NIOFSDir use direct ByteBuffers?
> ---------------------------------------
>
>                 Key: LUCENE-2056
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2056
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-2056.patch
>
>
> I'm trying to test NRT performance, and noticed when I dump the thread stacks that the
darned threads often seem to be in {{java.nio.Bits.copyToByteArray(Native Method)}}... so
I wondered whether we could/should use direct ByteBuffers, and whether that would gain performance
in general.  We currently just use our own byte[] buffer via BufferedIndexInput.
> It's hard to test since it's likely platform specific, but if it does result in gains
it could be an easy win.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message