lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 王建新 <>
Subject Re: How to use lucene for high search performance ?
Date Mon, 28 Jul 2008 03:54:50 GMT
Thanks a lot.

I have an idea, Can I use lucene on a 64bits VM?
In the condition, I can load all index files to ram. Then no io operation, I can execute concurrent
search in thread pool.

Its performance will be better?

----- Original Message ----- 
From: "Michael McCandless" <>
To: <>
Cc: <>
Sent: Saturday, July 26, 2008 4:59 AM
Subject: Re: How to use lucene for high search performance ?

Let's move this thread to java-user (CC'd).

王建新 wrote:

> Thank you.
> If the index files are very big(10G), I cannot load them to ram in  
> one process.

Ahh OK.

> Shoud I use MutilSearcher to load index files with serval processes?
> How about its performance?

MultiSearcher alone doesn't really scale up -- it just lets you  
combine the results of many Searchables.

Maybe you mean ParallelMlultiSearcher?  That class uses a separate  
thread to search each Searchable, so if you are on a multi core/cpu  
machine that should give a net reduction in latency of each search  
(though I don't have any experience here!).

> by the way, I think only .frq and .tis files need to load in ram.
> And it can save some ram.

You mean you don't use any positions information?  Really the OS  
should do the right thing for you -- it should only cache into its IO  
cache those files that you actually use after which searches should be  


> roy
> ----- Original Message -----
> From: "Michael McCandless" <>
> To: <>
> Sent: Thursday, July 24, 2008 6:09 PM
> Subject: Re: How to use lucene for high search performance ?
> Try InstantiatedIndexWriter/Reader (under contrib/instantiated)?
> It consumes more RAM than the RAMDirectory approach, but is faster
> performance.
> Mike
> PS -- this sort of question should go to java-user in the future.
> 王建新 wrote:
>> Hi,
>>   If I use lucene to execute many search requests at one time, the
>> io operation will be the bottleneck of the performance.
>>   So I use RAMDirectory to avoid io operation.
>>   But I found RAMDirectory cannot raise the performance much if the
>> index is big( about 1.2G).
>>   Could anyone give me any advice to raise the performance for
>> concurrent search operation?
>>   Thanks.
>> roy
View raw message