lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Svensson <si...@devhost.se>
Subject Re: Lucene Backup
Date Wed, 12 Dec 2012 22:01:24 GMT
Hi,

Your questions are answered by the first page in the green paper.

      * Don't copy the write.lock file.
      * Always copy the segments.gen file.
      * For all other files, Lucene is "write once.” This makes doing
        incremental backups very easy: Simply compare the file names.
        Once a file is written, it will never change; therefore, if
        you've already backed up that file, there's no need to copy it
        again.
      * You can do the copying in Java, or you can take the filenames
        and launch a shell to run your favorite backup or file archiving
        utility, such as rsync, robocopy, cp, tar, or zip. However, take
        extra care to catch and handle any errors that these tools might
        encounter. For example, if you get a disk full error, then that
        will certainly lead to a corrupt backup image.
      * You can even throttle the IO usage of the backup program so that
        it doesn't interfere with ongoing searching or indexing. It
        really doesn't matter to Lucene how long your backup takes
        because your backup will always be a point-in-time copy;
        however, while the backup is running, it will prevent deletion
        of any files referenced by the point-in-time commit point. This
        means your index might temporarily use more disk space.

// Simon

On 2012-12-12 22:53, jcwrequests wrote:

> Simon thank you for your response.  I put together some code at https://github.com/jcwrequests/Lokad.CQRS.Contrib/blob/master/Lokad.Portable.Lucene/AtomicStorage/FileDocumentReaderWriter.cs
 where I implement what was described in the green paper under the BackUo method.  What I
am confused about is the files that are returned from the snapshot should they only be copied
once and do I always copy the segment.gen files for every back up or only once?
>
> Any help you can provide would be greatly appreciated.  I will probably try run some
tests that not only back up the files but restore them to another location then see if the
index is functional.
>
> Thanks again,
>
> A Plus,
>
> Jason Wyglendowski
>
> Simon Svensson <sisve@devhost.se> wrote:
>
>> Hi,
>>
>> I just registered to retrieve the green paper, and it talks about
>> providing a SnapshotDeletionPolicy to the IndexWriter constructor, and
>> call SnapshotDeletionPolicy.Snapshot() to retrieve all files used by the
>> snapshot.
>>
>> There should be no problem at all porting the code present in the
>> article, except that it's for Lucene 2.3.1. You will probably find some
>> changes in casing when porting to Lucene.Net, and if you're using
>> Lucene.Net 3.0.3, you may also see some methods being replaced with
>> properties. There should be no showstoppers when porting the code.
>>
>> // Simon
>>
>> On 2012-12-12 01:06, jcwrequests wrote:
>>> I have read the hot backups green paper at http://www.manning.com/hatcher3/ and
was wondering if someone has an implementation using Lucene.net?  Are there anything special
with Lucene.net?
>>>
>>> Thanks,
>>>
>>> Jason Wyglendowski


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message