lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Becker <thomas.bec...@net-m.de>
Subject Re: lucene 2.9.0RC4 slower than 2.4.1?
Date Tue, 15 Sep 2009 15:23:47 GMT
Hey Mark,

yes. I'm running the app on unix. You see the difference between 2.9 and 2.4 here:

http://ankeschwarzer.de/tmp/graph.jpg

2.4 responds much quicker thus increasing throughput severly. I'm having a
single segment only:

-rw-r--r-- 1 asuser asgroup         20 Sep  9 16:40 segments.gen
-rw-r--r-- 1 asuser asgroup        294 Sep  9 16:44 segments_1o
-rw-r--r-- 1 asuser asgroup 2810603184 Sep  9 16:44 _7c.cfs

The FileChannel.read hotspot is indeed strange. Especially if taking into
account that the index is lying on a tmpfs (in memory). And 2.4 should be using
NIOFSDirectory as well?! Will check that.

Thanks a lot for your support!

Cheers,
Thomas

Mark Miller wrote:
> A few quick notes -
> 
> Lucene 2.9 old api doesn't appear much worse than Lucene 2.4?
> 
> You save a lot with the new Intern impl, because thats not a hotspot
> anymore. But then,
> RandomAccessFile seeks end up being a lot more of the pie. They look
> fairly similar in speed overall?
> 
> It looks like the major bottleneck with 2.9 new api is that its using
> NIOFSDirectory (your on unix I guess, and it now
> defaults to that on non Windows os's), and that appears to be a real
> killer for you. Its taking half the time for its
> reads.  ???
> 
> No conclusions yet, but I'm looking it over. Some other guys will come
> in with some ideas as well.
> 
> Do confirm that those profiling results are on a single segment though.
> 
> - Mark
> 
> 
> Mark Miller wrote:
>> Thomas Becker wrote:
>>   
>>> Here's the results of profiling 10 different search requests:
>>>
>>> http://ankeschwarzer.de/tmp/lucene_24_oldapi.png
>>> http://ankeschwarzer.de/tmp/lucene_29_oldapi.png
>>> http://ankeschwarzer.de/tmp/lucene_29_newapi.png
>>>
>>> But you already gave me a good hint. The index being used is an old one build
>>> with lucene 2.4. I will now try a freshly build 2.9 index and see if performance
>>> improves. Maybe that already solves the issue...stupid me...
>>>   
>>>     
>> That shouldn't be an issue unless there is some odd bug.
>>
>>   
>>> We're updating the index every 30 min. at the moment and it gets optimized after
>>> each update.
>>>   
>>>     
>> So this profiling is on an optimized index (eg a single segment) ?
>> That would be odd indeed, and possibly point to some of the scoring changes.
>>
>>   
>>> Mark Miller wrote:
>>>   
>>>     
>>>> Thomas Becker wrote:
>>>>     
>>>>       
>>>>> Hey Mark,
>>>>>
>>>>> thanks for your reply. Will do. Results will follow in a couple of minutes.
>>>>>
>>>>>
>>>>>   
>>>>>       
>>>>>         
>>>> Thanks, awesome.
>>>>
>>>> Also, how many segments (approx) are in your index? If there are a lot,
>>>> have you/can you try the same tests on an optimized index? Don't want to
>>>> get ahead of the profiling results, but just to continue the info loop.
>>>>
>>>>     
>>>>       
>>>   
>>>     
>>
>>   
> 
> 

-- 
Thomas Becker
Senior JEE Developer

net mobile AG
Zollhof 17
40221 Düsseldorf
GERMANY

Phone:    +49 211 97020-195
Fax:      +49 211 97020-949
Mobile:   +49 173 5146567 (private)
E-Mail:   mailto:thomas.becker@net-m.de
Internet: http://www.net-m.de

Registergericht:  Amtsgericht Düsseldorf, HRB 48022
Vorstand:         Theodor Niehues (Vorsitzender), Frank Hartmann,
                 Kai Markus Kulas, Dieter Plassmann
Vorsitzender des
Aufsichtsrates:   Dr. Michael Briem

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message