lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: [Solr Wiki] Update of "TaskList" by YonikSeeley
Date Thu, 10 Aug 2006 00:22:03 GMT
On 8/9/06, Yonik Seeley <yonik@apache.org> wrote:
> On 8/9/06, Chris Hostetter <hossman_lucene@fucit.org> wrote:
> > : + * NTFS w/ WinXP or later does support hard links for files (and cygwin
> > : "ln" works for files).  The current replication scripts could be ported
> > : to cygwin.
> >
> > Interesting.  THe one other potential hitch that comes to mind is the file
> > locking/deletion issue that forces Lucene to use a "deletable" file on
> > Windows -- I wonder if there would be any problematic interactions between
> > that code path and hardlinks?
>
> Yes, the behavior seems different (under cygwin at least).  I can't
> get an open file to delete, even if it's a hard link to the original
> file opened.
>
> The rsync dist method relies on being able to delete open files and
> replace them with new versions.  Bummer.

Although all is not lost.  AFAIK, the files that change are read
completely when the IndexReader is opened and then closed (segments,
.del, deletable).  So one would be left with two problems:

1) switching to the new index... you can't remove or mv a directory in
use AFAIK.  Solr would need to be changed to open a new IndexReader on
the newest snapshot directory (or something to that effect).

2) cleaning up old segment files... same problem that Lucene has now.
 Lucene tries to delete files in deletable during a segment merge I
think... since a "slave" doesn't do merges, a little piece of separate
code would need to try and delete the deletable files periodically.
That could be completely outside Solr though.

If someone want's to tackle using Windows with this index replication
method, it does seem doable though.

-Yonik

Mime
View raw message