lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Lucene segment selection strategy
Date Sat, 18 Jul 2015 11:59:03 GMT
This can also not happen anymore in Lucene 5: The segments_xy files are atomically renamed
from a temporary file name to the final name as last action of the commit. So the whole thing
is safe now. See also my talk about Java 7's NIO.2 on Berlinbuzzwords: https://www.youtube.com/watch?v=mARACndILQc

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> Sent: Saturday, July 18, 2015 12:43 AM
> To: Lucene Users; cooney.geoff@gmail.com
> Subject: Re: Lucene segment selection strategy
> 
> Curious ... Lucene should try to fallback to the older segments_N files (even
> if segments.gen points to the new, broken ones).
> 
> We've removed segments.gen as of 5.x and I think it's unlikely we'll do
> another 4.10.x release at this point, but maybe still open the issue in case
> others hit it?
> 
> Mike McCandless
> 
> http://blog.mikemccandless.com
> 
> 
> On Fri, Jul 17, 2015 at 4:37 PM, Geoff Cooney <cooney.geoff@gmail.com>
> wrote:
> > Hi,
> >
> > We recently had an issue with an index where two sequential aborted
> > but unsuccessfully rolled back commits resulted in empty segments_n
> > files, segments_i13p and segments_i13q in this case.  This resulted in
> > an exception whenever we tried to open the index until we manually
> > removed the bad segments_n files.
> >
> > Since the commits were never completed, segments.gen still pointed to
> > segments_i13o.  But it looks like lucene takes the more recent
> > generation it sees in the file list and never tries to open
> > segments_i13o(which is openable/uncorrupted in this case).
> >
> > My questions is if this is something I should file as a bug?  The
> > index is stored in a remote key-value store.  While I have a test that
> > recreates the issue with an NIOFSDirectory and artificially generated
> > exceptions, I'm not sure how likely it is that this scenario would
> > actually occur with a local disk index.
> >
> > We are running lucene 4.3.
> >
> > Thanks,
> > Geoff
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message