lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: Lucene 2.1, soon
Date Fri, 19 Jan 2007 00:42:30 GMT

On Jan 18, 2007, at 2:59 PM, Doron Cohen wrote:

> To my understanding the only remaining issue with NFS is: a reader
> might get an IO exception in case writer removed an old file that
> the reader is using.
> It is not a possible corruption that we try to solve, right?
> For that I think it is not worth to add that stuff again.

I agree, Doron.

I'd rather leave NFS as a problem case.

Now... how about having Readers establish advisory read locks when  
the operating system supports them?  That seems to me to be still in  
the spirit having Readers be read-only.

Then our problem set is reduced even further: only NFS systems using  
protocols prior to version 4.

It's probably not even worth it to perform the lock test I proposed  
earlier.  We just use file systems the way they're suppose to behave  
and eventually NFS catches up.

Provisionally, this is what I intend implement for KS, unless  
something better emerges from ongoing discussion.

> A writer's "two steps" policy - delete only files that
> "would have not been in use unless a reader did not refresh for X  
> minutes"
> is "fair enough" I think.
> By "two steps" I mean, start measuring time not from when segment  
> to be
> deleted was created, but rather from when its "next generation" was
> created.

Deletions are processed shortly after a new segments_N file gets  
written (at least on KS, and IIRC also for Lucene).  You'd always  
have to leave deletes to the next commit.

Marvin Humphrey
Rectangular Research

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message