lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Lea" <ian....@gmail.com>
Subject Re: Index optimization ...
Date Wed, 30 Jul 2008 14:33:16 GMT
If I was you I'd certainly try cutting the optimize frequency.  An
optimized index should indeed give the best search performance, but in
my experience it's generally plenty fast enough anyway, and I think
you said earlier that you were prepared to sacrifice a bit of search
or indexing speed.

Sorry, can't help with the questions on speeding it up except to
advise that you play with it.  Different indexes on different hardware
with different combinations of CPU/memory/disk can give different
results.

--
Ian.


On Wed, Jul 30, 2008 at 3:15 PM, Dragon Fly <dragon-fly999@hotmail.com> wrote:
> My understanding is that an optimized index gives the best search performance.  I can
change my configuration to optimize the index every 24 hours.  However, I still would like
to know if there is a way to speed up optimization by tweaking parameters like the merge factor.
>
>> Date: Wed, 30 Jul 2008 15:03:37 +0100
>> From: ian.lea@gmail.com
>> To: java-user@lucene.apache.org
>> Subject: Re: Index optimization ...
>>
>> OK, but why do you need to optimize before every swap?  Have you tried
>> with less frequent optimizes?
>>
>> --
>> Ian.
>>
>>
>> On Wed, Jul 30, 2008 at 3:00 PM, Dragon Fly <dragon-fly999@hotmail.com> wrote:
>> > I have two copies (active/inactive) of the index.  Searches are executed against
the "active" index and new documents get added to the "inactive" copy.  The two indexes get
swapped every 4 hours (so that new documents are visible to the end user).  Optimization is
done before the inactive copy is made active.
>> >
>> >> Date: Wed, 30 Jul 2008 14:54:03 +0100
>> >> From: ian.lea@gmail.com
>> >> To: java-user@lucene.apache.org
>> >> Subject: Re: Index optimization ...
>> >>
>> >> Why do you run an optimize every 4 hours?
>> >>
>> >>
>> >> --
>> >> Ian.
>> >>
>> >>
>> >> On Wed, Jul 30, 2008 at 2:46 PM, Dragon Fly <dragon-fly999@hotmail.com>
wrote:
>> >> > Perhaps I didn't explain myself clearly so please let me try it again.
 I'm happy with the search/indexing performance.   However, my index gets fully optimized
every 4 hours and the time it takes to fully optimize the index is longer than I like.  Is
there anything that I can do to speed up the optimization? I don't fully understand the different
parameters (e.g. merge factor).  If I decrease the merge factor, would it make the indexing
slower (which I'm OK with) but the optimization faster? Thank you.
>> >> >
>> >> >> Date: Tue, 29 Jul 2008 08:32:46 +0200
>> >> >> From: asbjorn@fellinghaug.com
>> >> >> To: java-user@lucene.apache.org
>> >> >> Subject: Re: Index optimization ...
>> >> >>
>> >> >> John Griffin:
>> >> >> > Use IndexWriter.setRAMBufferSizeMB(double mb) and you won't
have to
>> >> >> > sacrifice anything. It defaults to 16.0 MB so depending on
the size of your
>> >> >> > index you may want to make it larger. Do some testing at various
values to
>> >> >> > see where the sweet spot is.
>> >> >> >
>> >> >>
>> >> >> Also, have a look at
>> >> >> http://wiki.apache.org/lucene-java/ImproveIndexingSpeed, which
provides
>> >> >> a range of helping advices in terms of enhanced indexing speed.
>> >> >>
>> >> >> --
>> >> >> Asbjørn A. Fellinghaug
>> >> >> asbjorn@fellinghaug.com
>> >> >>
>> >> >> ---------------------------------------------------------------------
>> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >> >>
>> >> >
>> >> > _________________________________________________________________
>> >> > With Windows Live for mobile, your contacts travel with you.
>> >> > http://www.windowslive.com/mobile/overview.html?ocid=TXT_TAGLM_WL_mobile_072008
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >
>> > _________________________________________________________________
>> > With Windows Live for mobile, your contacts travel with you.
>> > http://www.windowslive.com/mobile/overview.html?ocid=TXT_TAGLM_WL_mobile_072008
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> _________________________________________________________________
> With Windows Live for mobile, your contacts travel with you.
> http://www.windowslive.com/mobile/overview.html?ocid=TXT_TAGLM_WL_mobile_072008

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message