lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Lucene index integrity during a system crash
Date Sat, 16 Jul 2005 22:22:02 GMT
Hi Jian,

Yes, I think what you describes is correct.  You may end up with some
"junk index segments" in the index directory, but as long as they are
not recorded in segments file, they are irrelevant.

Otis
P.S.
Did you ask you locking in Lucene the other day?


--- jian chen <chenjian1227@gmail.com> wrote:

> Hi, Otis,
> 
> Thanks for your email. As this is very important for using Lucene in
> our production system, I looked at the code to try to understand.
> Here
> is my observation why the index won't be corrupted during a system
> crash.
> 
> In the IndexWriter.java mergeSegments(...) method, there are two
> lines:
> segmentInfos.write(directory);	  // commit before deleting
> deleteSegments(segmentsToDelete);//delete unused segments
> 
> The sgementInfos.write(...) writes the new segments file as
> "segments.new", once the write is complete, it renames "segments.new"
> to "segments".
> 
> I guess the rename operation is atomic as guaranteed by the operating
> system. Otherwise, the "segments" file will be left in an
> inconsistent
> state during the system crash.
> 
> It also appears to me that the "segments" file is the single point to
> switch from old set of index segments to new ones. In case of a
> system
> failure, the old "segments" file will be used anyway, so, no
> corruption.
> 
> Is this understanding correct and thorough?
> 
> Thanks a lot,
> 
> Jian
> 
> On 7/16/05, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> > The only corruption that I've seen mentioned on this list so far
> was
> > the corruption of the segments file, and even that people have been
> > able to manually edit with a hex editor.
> > 
> > Otis
> > 
> > 
> > --- jian chen <chenjian1227@gmail.com> wrote:
> > 
> > > Hi,
> > >
> > > I know Lucene does not have transaction support at this stage.
> > > However, I want to know what will happen if there is an operating
> > > system crash during the indexing process, will the Lucene index
> got
> > > corrupted?
> > >
> > > Thanks,
> > >
> > > Jian
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> > 
> > 
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> > 
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message