lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Masurel <paul.masu...@gmail.com>
Subject Lucene commit
Date Mon, 22 Aug 2016 06:31:50 GMT
Hi,

If I understand correctly, Lucene indexing threads are working on their own
individual segment.
When a thread has enough documents in its segment, it flushes it on disc
and starts a new one.
But segments are only searchable when they are commited.

Now my question is, wouldn't it be nice to be able to set up Lucene so that
segments are made searchable as soon as they are flushed?

Commit would still play the roll of "checkpoint" in a hardware failure
scenario.
This is different from the old "autocommit" feature in that sense.

Of course, this "searchable yet not committed flushed segment" leads to the
following weird behavior :
- documents can become searchable and in case of failure, become not
searchable
(and then eventually searchable again if the client does its job properly
and reindexes rollbacked documents).
- one document can become searchable after another one even though it was
added before.

The benefit would be to reduce the average latency for a document
to become searchable, without hurting throughput by calling commit() too
frequently.

Regards,

Paul

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message