lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhang, Lisheng" <Lisheng.Zh...@BroadVision.com>
Subject RE: In lucene 2.3.2, needs to stop optimization?
Date Fri, 24 Sep 2010 16:12:56 GMT
Hi,

Thanks very much, I definitely plan to upgrade lucene.

I did not keep IndexWriter open partly because in our app we
have more than 3K independent lucene directories, so it would
be hard to put them all into memory, but I may cache some
busiest ones.

Best regards, Lisheng

-----Original Message-----
From: Danil TORIN [mailto:torindan@gmail.com]
Sent: Friday, September 24, 2010 12:01 AM
To: java-user@lucene.apache.org
Subject: Re: In lucene 2.3.2, needs to stop optimization?


Is it possible for you to migrate to 2.9.x ? Or even 3.x ?
There are some huge optimization in 2.9 on reopening indexes that
significantly improve search speed.

I'm not sure..but I think indexWriter.getReader() for almost realtime
was added to 2.9, so you can keep your writer always open and get very
cheaply a new reader on each search request.

On Fri, Sep 24, 2010 at 09:47, Zhang, Lisheng
<Lisheng.Zhang@broadvision.com> wrote:
> Hi,
>
> I read document/code and did some experiments, one possibility
> is to raise mergeFactor to high value, say close to 2Billion,
> then a lot of small files are created and after >500 docs are
> created separately, search speed dropped sharply.
>
> I noticed with our current data, if I add one doc then call
> optimize(), it took about 7s, this is too slow for real time
> search.
>
> If I keep mergeFactor as 10 and donot call optimize(), does it
> mean from time to time IndexWriter would optimize on background,
> when it happens, it may take a few seconds (so Index will delay
> a few seconds)?
>
> Should I use high mergeFactor and optimize once a day, or use
> default mergeFactor and donot call optimize? maybe latter is
> better, but I am concerned about occasional slowness?
>
> Currently I donot plan to keep IndexWriter constantly open, but
> open/close for each index request.
>
> Any suggestion to improve would be appreciated,
>
> Lisheng
>
> -----Original Message-----
> From: Zhang, Lisheng [mailto:Lisheng.Zhang@BroadVision.com]
> Sent: Thursday, September 23, 2010 6:11 PM
> To: java-user@lucene.apache.org
> Subject: In lucene 2.3.2, needs to stop optimization?
>
>
> Hi,
>
> We are using lucene 2.3.2, now we need to index each document as
> fast as possible, so user can almost immediately search it.
>
> So I am considering stop IndexWriter optimization during real time,
> then in relatively off-time like late night we may call IndexWriter optimize
> method explicitly once.
>
> What is the most efficient way to completely turn off IndexWriter merge
> in lucene 2.3.2?
>
> Thanks very much for helps, Lisheng
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Mime
View raw message