lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kelvin Tan <lists-kel...@relevanz.com>
Subject RE: Disappearing segments
Date Mon, 03 May 2004 04:57:51 GMT
<snip>
> you backup the index during an indexing you could end up with a limp index
> missing a few files, hence the missing segments, I would check for write and
> commit locks pre-backup so as to avoid that. 

That's what I needed to know! Thanks!

Kelvin

> -----Original Message-----
> From: Kelvin Tan [mailto:lists-kelvin@relevanz.com]
> Sent: Monday, May 03, 2004 6:52 AM
> To: Lucene Users List
> Subject: RE: Disappearing segments
> 
> 
> Thanks for responding Nader.
> 
> hmmmm...you've hit the nail on the spot. I do have a cron job which backs up
> the
> index. Its run in a batch index scheduled job.
> 
> The logic is basically
> 
> backupindex()
> try
> {
> batchindex()
> }
> catch(Exception e)
> {
> deleteindex();
> copyfrombackuptoindex()
> deletebackup();
> }
> 
> I assume that the original index before backing up was complete and
> 'working'.
> I'm also deleting the index that failed, instead of just overwriting. Where
> did
> I go wrong?
> 
> I'm not checking that the index isn't write-locked before backing up, but I
> don't think that's the problem (though it very well can be a separate
> problem).
> 
> Kelvin
> 
> On Fri, 30 Apr 2004 23:20:42 +0400, Nader Henein said:
>> Could you share you're indexing code, and just to make sure id there
>> anything running on your machine that could delete these files, like
>> an a cron job that'll back up the index.
>> 
>> You could go by process of elimination and shut down your server and
>> see if the files disappear, coz if the problem is contained within the
>> server you know that you can safely go on the DEBUG rampage.
>> 
>> Nader
>> 
>> -----Original Message-----
>> From: Kelvin Tan [mailto:lists-kelvin@relevanz.com]
>> Sent: Friday, April 30, 2004 9:15 AM
>> To: Lucene Users List
>> Subject: Re: Disappearing segments
>> 
>> An update:
>> 
>> Daniel Naber suggested using IndexWriter.setUseCompoundFile() to see
>> if it happens with the compound index format. Before I had a chance to
>> try it out, this happened:
>> 
>> java.io.FileNotFoundException: C:\index\segments (The system cannot
>> find the file specified) at java.io.RandomAccessFile.open(Native
>> Method) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)
>> at
>> org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.j
>> ava:321)
>> at
>> org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:329)
>> at
>> org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268)
>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:71)
>> at
>> org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:154)
>> at org.apache.lucene.store.Lock$With.run(Lock.java:116)
>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:149)
>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:131)
>> 
>> so even the segments file somehow got deleted. Hoping someone can shed
>> some light on this...
>> 
>> Kelvin
>> 
>> On Thu, 29 Apr 2004 11:45:36 +0800, Kelvin Tan said:
>>> Errr, sorry for the cross-post to lucene-dev as well, but I realized
>>> this mail really belongs on lucene-user...
>>> 
>>> I've been experiencing intermittent disappearing segments which
>>> result in the following stacktrace:
>>> 
>>> Caused by: java.io.FileNotFoundException: C:\index\_1ae.fnm (The
>>> system cannot find the file specified) at
>>> java.io.RandomAccessFile.open(Native Method) at
>>> java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)
>>> at
>>> org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.j
>>> a
>>> va:321) at
>>> org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:329)
>>> at org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268)
>>> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:78)
>>> at
>>> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:104)
>>> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:95)
>>> at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:112)
>>> at org.apache.lucene.store.Lock$With.run(Lock.java:116)
>>> at org.apache.lucene.index.IndexReader.open(IndexReader.java:103)
>>> at org.apache.lucene.index.IndexReader.open(IndexReader.java:91)
>>> at
>>> org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:75)
>>> 
>>> The segment that disappears (_1ae.fnm) varies.
>>> 
>>> I can't seem to reproduce this error consistently, so don't have a
>>> clue what might cause it, but it usually happens after the
>>> application has been running for some time. Has anyone experienced
>>> something similar, or can anyone point
>> me
>>> in the right direction?
>>> 
>>> When this occurs, I need to rebuild the entire index for it to be
>>> usable. Very troubling indeed...
>>> 
>>> Kelvin
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message