lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Angelov, Rossen" <Rossen.Ange...@SourceMedia.com>
Subject RE: how long should optimizing take
Date Thu, 02 Jun 2005 19:31:23 GMT
Thanks for the suggestion, Jian Chen's idea is very similar too.
Probably optimizing that often is not necessary and not that critical for
speeding up the searches.

I'll try changing the index process not to optimize at all and execute the
optimization independently of the indexing on a weekly bases.

Ross

-----Original Message-----
From: Dan Armbrust [mailto:daniel.armbrust.list@gmail.com]
Sent: Thursday, June 02, 2005 11:10 AM
To: java-user@lucene.apache.org
Subject: Re: how long should optimizing take


I would run your optimize process in a separate thread, so that your web 
client doesn't have to wait for it to return.

You may even want to set the optimize part up to run on a weekly 
schedule, at a low load time.  I probably wouldn't reoptimize after 
every 30 documents, on an index that size.

Optimizing takes a while on your index, because it basically has to copy 
the entire index to a new index, so it will take how ever long it takes 
to copy 2 GB's on your hardware + a small amount of overhead...

Dan

Angelov, Rossen wrote:

>I would like to bring that issue up again as I haven't resolved it yet and
>haven't found what's causing it.
>
>Any help, ideas or sharing experience are welcome!
>
>Thanks,
>Ross
>
>-----Original Message-----
>From: Angelov, Rossen 
>Sent: Friday, May 27, 2005 10:42 AM
>To: 'java-user@lucene.apache.org'
>Subject: how long should optimizing take
>
>
>Hi,
>
>I'm having problems with the Lucene optimization. Two of the indexes are
>about 2BG big and every day about 30 documents are added to each of these
>indexes. At the end of the indexing the IndexWriter optimize() method is
>executed and it takes about 30 minutes to finish the optimization for each
>index.
>
>The indexing happens through a web service. A servlet takes an http request
>and executes methods to index the new documents and optimize the indexes.
>
>The problem is that the request takes too long to finish because of the
>optimization and the web server doesn't return a response. The browser will
>keep waiting forever.
>
>Has anybody else experienced similar behavior with the optimization
process?
>
>Thanks,
>Ross
>
>"This communication is intended solely for the addressee and is
>confidential and not for third party unauthorized distribution."
>
>
>
>"This communication is intended solely for the addressee and is
>confidential and not for third party unauthorized distribution."
>
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

"This communication is intended solely for the addressee and is
confidential and not for third party unauthorized distribution."


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message