lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ning Li <nin...@us.ibm.com>
Subject Re: [jira] Commented: (LUCENE-565) Supporting deleteDocuments in IndexWriter (Code and Performance Results Provided)
Date Thu, 06 Jul 2006 19:07:20 GMT
Hi Otis and Robert,

I added an overview of my changes in JIRA. Hope that helps.

> Anyway, my test did exercise the small batches, in that in our
> incremental updates we delete the documents with the unique term, and
> then add the new (which is what I assumed this was improving), and I
> saw o appreciable difference.

Robert, could you describe a bit more how your test is set up? Or a short
code snippet will help me explain.

Without the patch, when inserts and deletes are interleaved in small
batches, the performance can degrade dramatically because the ramDirectory
is flushed to disk whenever an IndexWriter is closed, causing a lot of
small segments to be created on disk, which eventually need to be merged.

Is this how your test is set up? And, what are the maxBufferedDocs and the
maxBufferedDeleteTerms in your test? You won't see a performance
improvement
if they are about the same as the small batch size. The patch works by
internally buffering inserts and deletes into larger batches.

Regards,
Ning


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message