lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Engels" <reng...@ix.netcom.com>
Subject RE: [jira] Commented: (LUCENE-414) Java NIO patch against Lucene 1.9
Date Wed, 26 Oct 2005 22:08:27 GMT
You are correct, this is to get around JDK bug 6265734. (The bug was
originally cited by me, by the test code attached to the bug seems to bear
out that my assessment is correct). It should be document in the code that
this is a work-around (and does increase the number of file handles needed).
I will look into whether or not using multiple RandomAccessFiles has any
performance difference.

I am not sure how to benchmark this. I state this from my understanding of
optimizing disk subsystems, but I am sure it is very hardware dependent. I
do know by reading the SCSI documentation, and other UltaATA documentation
that the controller will coalesce requests, so you need to get multiple
requests to the controller. If the thread blocks in java, you will never get
multiple requests to the controller.

I am working on a performance test case for the caching right now...



-----Original Message-----
From: Doug Cutting [mailto:cutting@apache.org]
Sent: Wednesday, October 26, 2005 4:51 PM
To: java-dev@lucene.apache.org
Subject: Re: [jira] Commented: (LUCENE-414) Java NIO patch against
Lucene 1.9


Robert Engels wrote:
> The reason for using Nio and not IO is IO requires multiple file handles
per file. There are already numerous bugs/work-arounds in Lucene to limit
the use of file handles (as this is a OS limited resource), so I did not
wish to further increase the number of file descriptors needed.

Yes, but it appears to me that the submitted NioFile class opens a new
file handle per channel.  So I don't see how this addresses that.

> Your statement that a raid system would be needed to exploit the added
concurrency is not exactly correct. By using multiple threads, even if the
disk is busy handling a request, the OS can combine the pending requests and
perform more efficient reads to the disk subsystem when it becomes
available.

Perhaps.  It would be nice to see a benchmark demonstrating this.

> I also dispute the performance numbers cited. In my testing the 'user
level' cache improved performance of query operations nearly 100%. I will
write a testcase to demonstrate the increased performance. This testcase can
be written independent of Lucene.

Can you provide your benchmark results?

Thanks,

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message