lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jian chen <chenjian1...@gmail.com>
Subject Re: Lucene index integrity during a system crash
Date Sun, 17 Jul 2005 00:10:26 GMT
Thanks Otis and Nikhil for your confirmation. I am more confident
about the Lucene index integrity.

Nikhil, I don't see th reason why there is a corrupted .fdx file.
Could it be caused by multi-threaded access to the index?

Otis, I don't remember I asked about locking questions the other day.
I think it should be another guy.

Thanks all,

Jian

On 7/16/05, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> Hi Jian,
> 
> Yes, I think what you describes is correct.  You may end up with some
> "junk index segments" in the index directory, but as long as they are
> not recorded in segments file, they are irrelevant.
> 
> Otis
> P.S.
> Did you ask you locking in Lucene the other day?
> 
> 
> --- jian chen <chenjian1227@gmail.com> wrote:
> 
> > Hi, Otis,
> >
> > Thanks for your email. As this is very important for using Lucene in
> > our production system, I looked at the code to try to understand.
> > Here
> > is my observation why the index won't be corrupted during a system
> > crash.
> >
> > In the IndexWriter.java mergeSegments(...) method, there are two
> > lines:
> > segmentInfos.write(directory);          // commit before deleting
> > deleteSegments(segmentsToDelete);//delete unused segments
> >
> > The sgementInfos.write(...) writes the new segments file as
> > "segments.new", once the write is complete, it renames "segments.new"
> > to "segments".
> >
> > I guess the rename operation is atomic as guaranteed by the operating
> > system. Otherwise, the "segments" file will be left in an
> > inconsistent
> > state during the system crash.
> >
> > It also appears to me that the "segments" file is the single point to
> > switch from old set of index segments to new ones. In case of a
> > system
> > failure, the old "segments" file will be used anyway, so, no
> > corruption.
> >
> > Is this understanding correct and thorough?
> >
> > Thanks a lot,
> >
> > Jian
> >
> > On 7/16/05, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> > > The only corruption that I've seen mentioned on this list so far
> > was
> > > the corruption of the segments file, and even that people have been
> > > able to manually edit with a hex editor.
> > >
> > > Otis
> > >
> > >
> > > --- jian chen <chenjian1227@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I know Lucene does not have transaction support at this stage.
> > > > However, I want to know what will happen if there is an operating
> > > > system crash during the indexing process, will the Lucene index
> > got
> > > > corrupted?
> > > >
> > > > Thanks,
> > > >
> > > > Jian
> > > >
> > > >
> > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > >
> > >
> > >
> > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message