lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <ysee...@gmail.com>
Subject Re: replication question
Date Tue, 16 Dec 2008 14:44:56 GMT
On Tue, Dec 16, 2008 at 1:04 AM, Michael Stoppelman <stopman@gmail.com> wrote:
> I've got a question from Doug's original email about replication (
> http://www.mail-archive.com/lucene-user@jakarta.apache.org/msg12709.html):
>
> "1. On the index master, periodically checkpoint the index. Every minute or
> so the IndexWriter is closed and a 'cp -lr index index.DATE' command is
> executed from Java, where DATE is the current date and time. This
> efficiently makes a copy of the index when its in a consistent state by
> constructing a tree of hard links. If Lucene re-writes any files (e.g., the
> segments file) a new inode is created and the copy is unchanged."
>
> Is closing the IndexWriter really a requirement on taking a snapshot? Or can
> one take a snapshot on an index being written, I've done this in my
> development environment and it seems to work fine w/o closing the
> IndexWriter.

There are subtle race conditions if you try to do this with a changing index.
At any instance in time, the index should be consistent, *but* you
can't actually make a snapshot instantaneously.

So this is doable, but it would require some complex retry logic like
IndexReader has when opening an index.

> Also the solr replication shell scripts don't seem to worry
> about this either.

Solr takes snapshots when it knows it's not updating the index (new
index changes are internally blocked when calling snapshooter).

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message