lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tate Avery" <tate.av...@nstein.com>
Subject RE: FileChannel implementation of Directory
Date Tue, 21 Oct 2003 17:03:28 GMT
Thank you Doug.

My next tests will be with RAID (RAID0, I believe).

My suspicion was that the OS was serializing the disk reads (from IDE or SCSI).  But, I wasn't
sure how to 'open the gates' for concurrent reads.

My index was fairly large (about 3 million docs) and my queries where random (since the corpus
was a statistically generated numeric one).  So, with some sort of RAID setup, I hope to see
the FileChannelDirectory beat the FSDirectory with concurrent search threads.

I will post a follow-up after the RAID test (hopefully next week).


Again, thank you.
Tate


-----Original Message-----
From: Doug Cutting [mailto:cutting@lucene.com]
Sent: October 21, 2003 12:28 PM
To: Lucene Developers List
Subject: Re: FileChannel implementation of Directory


Tate Avery wrote:
> Does anyone know why I might not be seing a performance increase for multiple concurrent
threads using my "FileChannelDirectory" ?

Are you using a RAID system?  If the data is already cached in RAM, then 
the i/o calls may be so fast that concurrency doesn't make things 
noticeably faster.  If the data is not already cached, then, with a 
single disk, the OS will have to serialize the i/o requests to that 
drive, so there's no opportunity for concurrency.  If however you have 
an index that is too large to be cached in RAM, your query stream is 
diverse enough so that it cannot be cached, and you have a RAID-based 
file system which can support multiple concurrent i/o operations, then 
you may see a speedup.  Does that make sense?

Doug


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message