lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jokin Cuadrado" <joki...@gmail.com>
Subject Re: IndexWriter.flush performance
Date Sun, 07 Dec 2008 13:27:23 GMT
Avery time you flush the index, you are writing a small index to the
disk. Theres a  defined value (mergefactor) that decides when it have
to merge all of those small index in a bigger one, so as the index
grown the merges are bigger. First  you merge 10 indexes of 1
document, then 10 indexes of 10 documents, then 10 indexes of 100
documents and so on. The flush (that becomes commit) it's a costly
operation, so  i suggest to refactor your logic so you can flush every
n documentes.  However, you can change the mergepolicy and do the
merges in background, so the flush operation exit quicky, but be aware
that the index is left working behind, so the performance will we very
bad.

On Sun, Dec 7, 2008 at 1:27 PM, mimounl <mimounl@hotmail.com> wrote:
>
> Hello,
>
> In my application, I need to flush data each time a modification is made. So
> each time an entry is added in the lucene index we call IndexWriter.flush
> this way all data are secured on file system.
>
> We noticed that this operation is more and more time consuming while the
> size of the index raises.
>
> How can I tune the index to have a duration constant for the flush operation
> ?
>
> Before changing our logic for example flushing each "n" entries, I'd like to
> be sure that there isn't something I miss.
>
> I saw that the IndexWriter API changed in last version but our version is
> 2.3.1 that don't have for example the commit operation (and the flush method
> is deprecated).
>
> Thank you for your suggestions.
> --
> View this message in context: http://www.nabble.com/IndexWriter.flush-performance-tp20880541p20880541.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message