lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doron Cohen <DOR...@il.ibm.com>
Subject Re: can we do partial optimization?
Date Tue, 04 Dec 2007 06:11:16 GMT
It doesn't make sense to optimize() after every document add.
Lucene in fact implements a logic in the spirit of what you
describe below, when it decides to merge segments on the fly.

There are various ways to tell Lucene how often to flush
recently added/updated documents and what to merge.

But it will pay to check the simple things first, like - Are
you closing and opening the index writer after each document
add (don't)? Are you deleting using IndexRedaer or
IndexWriter (use IndexWriter if you can)? etc.

It's a good start to going again through Lucene FAQ,
http://wiki.apache.org/lucene-java/LuceneFAQ
and in addition see this wiki page on performance:
http://wiki.apache.org/lucene-java/BasicsOfPerformance

Good luck, and let us know how it went!
Doron

"Nizamul" <nizamulh@rediff.co.in> wrote on 03/12/2007 10:48:34:

> Hello,
> I am very new to Lucene.I am facing one problem.
>  I have one very large index which is constantly getting
> update(add and delete) at a regular interval.after which I am
> optimizing the whole index (otherwise searches will be slow)
> but optimization takes time.So I was thinking to merge only the
> segments of lesser size(I guess it will be a good compromise
> between search time and optimization time) i.e. suppose I have
> 10 segment
> 1 of 10,000,000 doc
> 4 of 100,000 doc
> 4 of 10,000 doc
> and 1 of 5 doc.
>
> I want to merger 9 segment of lesser size  in to  one(I believe
> this would not take much time and searching will improve a
> lot).But I don't know how to do partial merging.Whether Lucene
> allow it or not?? or if I can extend indexWriter and add a
> method optimize of my own where I can specify which cfs file to
> chose for optimization?
>
> Thanks and Regards,
> Nizam


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message