Mailing-List: contact general-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
From: Michael McCandless <lucene@mikemccandless.com>
To: general@lucene.apache.org
In-Reply-To: <002401c8ed79$26610f60$0308a8c0@WangJianxin>
Subject: Re: How to use lucene for high search performance ?
References: <001701c8ed74$1c781480$0308a8c0@WangJianxin>
 <A3A4163E-E09E-4A05-BFC8-70A81EBCA2F3@mikemccandless.com>
 <002401c8ed79$26610f60$0308a8c0@WangJianxin>
Message-Id: <4AA55872-F1ED-41C9-920E-616CF2C02847@mikemccandless.com>
Content-Type: text/plain; charset=GB2312; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v926)
Date: Fri, 25 Jul 2008 16:59:50 -0400
Cc: java-user@lucene.apache.org


Let's move this thread to java-user (CC'd).

=CD=F5=BD=A8=D0=C2 wrote:

> Thank you.
>
> If the index files are very big(10G), I cannot load them to ram in =20
> one process.

Ahh OK.

> Shoud I use MutilSearcher to load index files with serval processes?
> How about its performance?

MultiSearcher alone doesn't really scale up -- it just lets you =20
combine the results of many Searchables.

Maybe you mean ParallelMlultiSearcher?  That class uses a separate =20
thread to search each Searchable, so if you are on a multi core/cpu =20
machine that should give a net reduction in latency of each search =20
(though I don't have any experience here!).

> by the way, I think only .frq and .tis files need to load in ram.
> And it can save some ram.

You mean you don't use any positions information?  Really the OS =20
should do the right thing for you -- it should only cache into its IO =20=

cache those files that you actually use after which searches should be =20=

fast.

Mike

>
> roy
>
> ----- Original Message -----
> From: "Michael McCandless" <lucene@mikemccandless.com>
> To: <general@lucene.apache.org>
> Sent: Thursday, July 24, 2008 6:09 PM
> Subject: Re: How to use lucene for high search performance ?
>
>
>
> Try InstantiatedIndexWriter/Reader (under contrib/instantiated)?
>
> It consumes more RAM than the RAMDirectory approach, but is faster
> performance.
>
> Mike
>
> PS -- this sort of question should go to java-user in the future.
>
> =CD=F5=BD=A8=D0=C2 wrote:
>
>>
>> Hi,
>>   If I use lucene to execute many search requests at one time, the
>> io operation will be the bottleneck of the performance.
>>   So I use RAMDirectory to avoid io operation.
>>   But I found RAMDirectory cannot raise the performance much if the
>> index is big( about 1.2G).
>>   Could anyone give me any advice to raise the performance for
>> concurrent search operation?
>>   Thanks.
>>
>> roy