lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roger dimitri <rogerdimi...@yahoo.com>
Subject Re: backup RAMDirectory to file
Date Fri, 21 Mar 2008 17:46:08 GMT
Thank you so much Michael and Grant for your suggestions. I haven't tried SnapshotDeletionPolicy
yet (thanks for the hint, I will do it now) and I guess Incremental back up may not work in
my case since I have periodic Index cleaning jobs in RAMDirectory.

Thanks again,
Roger 

----- Original Message ----
From: Michael McCandless <lucene@mikemccandless.com>
To: java-user@lucene.apache.org
Sent: Friday, March 21, 2008 4:24:13 AM
Subject: Re: backup RAMDirectory to file


I think both the original approach and addIndexes below will work  
here, though the original approach should be faster.

But, there are some caveats.  You have to make sure you do the backup  
with the writer on the ramDir closed.  If there is a writer open, it  
could be changing files during your backup which will corrupt the  
backup image.

Alternatively, you could do a true "hot backup" (not having to close  
the writer on the ramDir) by using SnapshotDeletionPolicy.

You may want to do an incremental backup of the index.  Since Lucene  
is write-once, you only have to look for new filenames to copy (old  
filenames will never be changed).

Mike

Grant Ingersoll wrote:
> I think you could try:
>
> IndexWriter writer = new IndexWriter(fileDirectory, ...)
> writer.addIndexes(ramDir)
>
> -Grant
>
> On Mar 20, 2008, at 2:47 PM, roger dimitri wrote:
>
>> Hi,
>>  I am using the Directory class's copy method to periodically sync  
>> my RAM based index to a file based index that's supposed to serve  
>> as a hot backup. I want to know if this is the right way to  
>> maintain a periodic backup of my RAM based index and, if Yes, then  
>> is it reliable in a production instance with almost 12 to 15 GB of  
>> Lucene index loaded into the RAMDirectory.
>> Any advice is greatly appreciated.
>> I have a sample working code snippet here:
>>
>> public class RAMtoFILEcopy_test1 {
>>
>>
>>    static String[] queries = {
>>        "a OR b AND c",
>>        "(a OR b) AND c",
>>        "a OR (b AND c)",
>>        "a AND b",
>>        "a AND b OR c AND d",
>>        "(a AND b) OR (c AND d)",
>>        "a AND (b OR c) AND d",
>>        "((a AND b) OR c) AND d",
>>        "a AND (b OR (c AND d))",
>>        "a AND b AND c AND d",
>>
>>        "a OR b AND NOT c",
>>        "(a OR b) AND NOT c",
>>        "a OR (b AND NOT c)",
>>        "a AND NOT d",
>>        "a AND NOT b OR c AND NOT d",
>>        "(a AND NOT b) OR (c AND NOT d)",
>>        "a AND NOT (b OR c) AND NOT d",
>>        "((a AND NOT b) OR c) AND NOT d",
>>        "a AND NOT (b OR (c AND NOT d))",
>>        "a AND NOT b AND NOT c AND NOT d",
>>
>>    "a OR NOT b",
>>    "a OR NOT a",
>>
>>    "a b",
>>    "a b c",
>>    "a b (c d e)",
>>    "+a +b",
>>    "a -b",
>>    "a +b -c",
>>    "+a b -c",
>>    "+a -b c",
>>    "a -b -c",
>>    "-a b c",
>>
>>    "a OR b c AND d",
>>    "a OR b c",
>>    "a AND b c",
>>    "a OR b c OR d",
>>    "a OR b c d OR e",
>>    "a AND b c AND d",
>>    "a AND b c d AND e"
>>    };
>>
>>    public static void main(String argv[]) throws Exception {
>>        Directory dir = new RAMDirectory();
>>        Analyzer analyzer = new StandardAnalyzer();
>>
>>        IndexWriter writer = new IndexWriter(dir, analyzer, true);
>>
>>        for ( int i=0; i < queries.length; i++ ) {
>>            Document doc = new Document();
>>            doc.add(new Field("text", queries[i], Field.Store.YES,  
>> Field.Index.TOKENIZED));
>>            writer.addDocument(doc);
>>        }
>>        writer.close();
>>
>>
>>        Directory dir2 = FSDirectory.getDirectory("/tmp/ 
>> TestHotbackup", true);
>>        dir.copy(dir, dir2, true);
>>
>>    }
>>
>> }
>>
>>
>> Thanks a lot,
>> Roger
>>
>>
>>
>>
>>       
>> _____________________________________________________________________ 
>> _______________
>> Never miss a thing.  Make Yahoo your home page.
>> http://www.yahoo.com/r/hs
>
> --------------------------
> Grant Ingersoll
> http://www.lucenebootcamp.com
> Next Training: April 7, 2008 at ApacheCon Europe in Amsterdam
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org







      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message