lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: lucene 2.9.0RC4 slower than 2.4.1?
Date Tue, 15 Sep 2009 18:18:37 GMT
The results:

config: impl=SeparateFile serial=false nThreads=4 iterations=100
bufsize=1024 poolsize=2 filelen=730554368
answer=-282295611, ms=173550, MB/sec=1683.7899579371938

config: impl=ChannelFile serial=false nThreads=4 iterations=100
bufsize=1024 poolsize=2 filelen=730554368
answer=-282295361, ms=1377768, MB/sec=212.09793463050383

config: impl=ChannelPread serial=false nThreads=4 iterations=100
bufsize=1024 poolsize=2 filelen=730554368
answer=-282295361, ms=632253, MB/sec=462.19115955163517

config: impl=PooledPread serial=false nThreads=4 iterations=100
bufsize=1024 poolsize=2 filelen=730554368
answer=-282295361, ms=774664, MB/sec=377.2238637654518

ClassicFile was heading for the same fate as ChannelFile.


I'll have to check what its like on the file system - but it appears
just ridiculously slower. Even with SeparateFile, All 4 cores are bouncing
from 0-12% independently and really favoring the low end of that.
ChannelPread appears no better.

There are results from other OS's/setups in the JIRA issue.

I'm using ext4.

Uwe Schindler wrote:
> How does a conventional file system compare?
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>   
>> -----Original Message-----
>> From: Mark Miller [mailto:markrmiller@gmail.com]
>> Sent: Tuesday, September 15, 2009 7:15 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: lucene 2.9.0RC4 slower than 2.4.1?
>>
>> Mark Miller wrote:
>>     
>>> Indeed - I just ran the FileReaderTest on a Linux tmpfs ramdisk - with
>>> SeparateFile all 4 of my cores are immediately pinned and remain so.
>>> With ChannelFile, all 4 cores hover 20-30%.
>>>
>>> It would appear it may not be a good idea to use NIOFSDirectory on
>>>       
>> ramdisks.
>>     
>>> Even still though - it looks like you have a further issue - your Lucene
>>> 2.9 old-api results don't use it, and are still not good.
>>>
>>>
>>>       
>> The quick results:
>>
>> ramdisk: sudo mount -t tmpfs tmpfs /tmp/space -o
>> size=1G,nr_inodes=200k,mode=01777
>>
>> config: impl=SeparateFile serial=false nThreads=4 iterations=100
>> bufsize=1024 poolsize=2 filelen=730554368
>> answer=-282295611, ms=173550, MB/sec=1683.7899579371938
>>
>> config: impl=ChannelFile serial=false nThreads=4 iterations=100
>> bufsize=1024 poolsize=2 filelen=730554368
>> answer=-282295361, ms=1377768, MB/sec=212.09793463050383
>>
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>     
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>   


-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message