lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin A. Burton" <bur...@newsmonster.org>
Subject Re: disable locks on read only indexes (performance improvement?)
Date Mon, 01 Dec 2003 22:30:17 GMT
Dror Matalon wrote:

>On Mon, Dec 01, 2003 at 01:38:48PM -0800, Kevin A. Burton wrote:
>  
>
>>Would there be any performance improvement in query throughput and 
>>latency if locking were disabled for readonly indexes?
>>
>>It doesnt' seem like it makes sense to worry about locking if you know 
>>for SURE that the index will NEVER be updated again. 
>>
>>I'm noticing this problem now.  We are running a live indexer which does 
>>a commit every N documents (right now 100,000) and then swaps the new 
>>index into the system live.  This index is never again updated and we 
>>use a multisearcher.  We then do index merges after a while into new 
>>indexes to keep performance high and reduce the number of indexes.
>>    
>>
>
>Sounds quite familiar. One question though, when you say "swaps" the new
>index, what do you mean? It's one area where locking might matter. If
>you just use a multisearcher and add the new index I'm guessing that it
>should work fine.
>
It's a safe operation.. It does a directory rename and then is added to 
the multi-searcher. It's sychronized.. and 100% safe ;)

>>I would assume that removing this lock could increase performance 
>>especially to allow multiple concurrent searches on the same data.
>>    
>>
>
>There was talk about providing that in an upcoming version. Until then
>you can try RODirectory:
>	http://www.csita.unige.it/software/free/lucene/
>  
>
Cool.. thanks.

-- 
    NewsMonster - http://www.newsmonster.org/

Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
       AIM - sfburtonator,  Web - http://www.peerfear.org/
GPG fingerprint: 4D20 40A0 C734 307E C7B4  DCAA 0303 3AC5 BD9D 7C4D
  IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message