lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Burleson <pburle...@gmail.com>
Subject Re: Swapping Indexes?
Date Tue, 17 Aug 2004 15:30:46 GMT
Forward back to list.


---------- Forwarded message ----------
From: Patrick Burleson <pburleson@gmail.com>
Date: Tue, 17 Aug 2004 11:30:19 -0400
Subject: Re: Swapping Indexes?
To: Stephane James Vaucher <vauchers@cirano.qc.ca>

Stephane,

Thank you for the ideas. I'm going about implenting idea 1 (I like the
idea of leaving the temp index around for recovery), but I have a
question reguarding your original index. Do you just copy over the
temp index and don't worry abou cleaning up the old index directory?

Right now I have my code deleting the files in the main index
directory after telling the search controller to switch to the temp
index. But by doing that, I need to manage existing searches and not
break them while they are running. I also still run into the open
files problem on Windows when trying to delete a file one of the
searchers has open before it's closed.

Thoughts?

Patrick




On Mon, 16 Aug 2004 18:22:20 -0400 (EDT), Stephane James Vaucher
<vauchers@cirano.qc.ca> wrote:
> I've tried two options that seem to work:
>
> 1) Have a singleton that is responsible that will control your searchers.
> This controller can temporarilly redirect your searchers to
> c:/temp/myindex, allowing you to copy you index to c:/myindex. After that
> process completes, your controller can tell your searchers to use
> c:/myindex, allowing you to then erase your temp index.
>
> If you index nightly, you can always *not* erase your tmp dir, your index
> process will do this automatically if you create your IndexWriter with
> the overwrite option. This way, you can have a backup index if there is
> a system failure at some point (like when you copy/move directories).
>
> 2) Use an incremental index. Regularly, I scan my files, see if there are
> modification/additions and update my master index. Removing from the
> master index, adding to a temp dir, then merging. I haven't seen any
> weirdness on windows with this process.
>
> HTH,
> sv
>
>
>
> On Mon, 16 Aug 2004, Patrick Burleson wrote:
>
> > I've read in the docs about updating an index and its suggestion
> > reguarding swapping out indexes with a directory rename.
> >
> > Here's my question, how to do this when searches are running live?
> >
> > Say I have a directory that holds the current valid index:
> >
> > C:\myindex
> >
> > and when I'm running my nightly process to generate the index, it gets
> > temporarily indexed to:
> >
> > C:\temp\myindex
> >
> > How can I very quickly replace C:\myindex with C:\temp\myindex?
> >
> > I can't simply do a rename since C:\myindex will likely have open
> > files. (Gotta love windows)
> >
> > And I can't delete all files in myindex, again because of the open files issue.
> >
> > Any ideas?
> >
> > Thanks,
> > Patrick
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message