lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject RE: Update performance/indexwriter.delete()?
Date Fri, 15 Apr 2005 00:18:57 GMT

You mentioned before that you can't "batch" your updates ... i can
understand not being able to batch updates by number of updates -- but why
can't you batch by time?

It may sound bad to only process updates once an hour, or once every half
hour, or once every 5 minutes, or even once every 30 seconds ... but if
you are truely processing your records in such rapid fire succession that
the cumulative (milli)seconds it takes to open/close the reader and
open/close the writer for each doc is expensive, then why can't you batch
on whatever that cumulative time durration is?

Why not write your updater such that waits at most N milliseconds for
updates to be sent to it, then as long as it recieved at least 1 doc: open
a reader, deletes all the matching docs, closes the reader, opens a
writer, add the new versions of the docs, close the writter.

then do some performance tests, and find the optimal value of N, so that
you are processing docs as fast you possibly can.



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message