lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Đạt Cao Mạnh <>
Subject Write buffering updates to temporary files?
Date Tue, 27 Dec 2016 22:51:34 GMT
Currently, we write buffering logs to current tlog and not apply that
updates to index. Then we rely on replay log to apply that updates to
index. But at the same time there are some updates also write to current
tlog and applied to the index.

For example, during peersync, if new updates come to replica we will end up
with this tlog
tlog : old1, new1, new2, old2, new3, old3
old updates belong to peersync, and these updates are applied to the index.
new updates belong to buffering updates, and these updates are not applied
to the index.

But writing all the updates to same current tlog make code base very
complex. Should we write buffering updates to another temporary file?

By doing this, it will help our code base simpler. It also makes replica
recovery for SOLR-9835 more easier. Because after peersync success we can
copy new updates from temporary file to current tlog, for example
tlog : old1, old2, old3
temporary tlog : new1, new2, new3
tlog : old1, old2, old3, new1, new2, new3

Note that in  SOLR-9835 we can not rely on fingerprint for peersync because
updates are not applied to replicas.

View raw message