lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Satuluri, Venu_Madhav" <Venu.Madhav.Satul...@deshaw.com>
Subject RE: Errors when searching index and writing to index simultaenously
Date Wed, 22 Mar 2006 14:16:46 GMT
I've figured out the problem:
1) I must make sure both the searching process and indexing process use
the same lock directory (Thanks, Luc!)
2) I must not execute Hits.doc() after closing IndexSearcher. i.e. I
must close IndexSearcher *after* I am done with retrieving documents
from Hits.

Thanks,
Venu

-----Original Message-----
From: Satuluri, Venu_Madhav 
Sent: Wednesday, March 22, 2006 7:36 PM
To: java-user@lucene.apache.org
Subject: RE: Errors when searching index and writing to index
simultaenously



> Make sure both the indexing process and the searcher process use the
> same directory to store the Lock files (default your home directory I
> believe).

I am not sure if earlier they were using the same directory. Now I've 
made sure they same directory; still I get the second type of exception
I mentioned in my earlier mail. That is, IndexSearcher is returning a
Hits
object, but Hits.doc() is throwing an exception ("Bad file number").
And, as I said, the index is getting corrupted whenever this happens.

Any ideas?

-Venu

-----Original Message-----
From: Satuluri, Venu_Madhav [mailto:Venu.Madhav.Satuluri@deshaw.com] 
Sent: woensdag 22 maart 2006 14:14
To: java-user@lucene.apache.org
Subject: Errors when searching index and writing to index simultaenously

Hi,

If I run IndexSearcher.search() at the same time an IndexWriter is
adding a document to the index, I get the following kind of exception
frequently:
java.io.FileNotFoundException: <indexDir>/_3j.fnm (No such file or
directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
        at
org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java
:376)
        at
org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:405)
        at
org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:268)
        at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:53)
        at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:109)
        at
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:94)
        at
org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:122)
        at org.apache.lucene.store.Lock$With.run(Lock.java:109)
        at
org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:95)
        at
org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:38)
        at
deshaw.reccms.indexer.LuceneSearcher.searchLucene(LuceneSearcher.java:10
6)
        at
deshaw.reccms.indexer.LuceneSearcher.main(LuceneSearcher.java:162)

At other times, the search() method returns the Hits object, but
Hits.doc(i) throws Exception. Whenever this happens, it seems that the
index is corrupted and is unusable forever.
Exception in thread "main" java.io.IOException: Bad file number
        at java.io.RandomAccessFile.seek(Native Method)
        at
org.apache.lucene.store.FSInputStream.readInternal(FSDirectory.java:415)
        at
org.apache.lucene.store.InputStream.refill(InputStream.java:158)
        at
org.apache.lucene.store.InputStream.readByte(InputStream.java:43)
        at
org.apache.lucene.store.InputStream.readBytes(InputStream.java:57)
        at
org.apache.lucene.index.CompoundFileReader$CSInputStream.readInternal(Co
mpoundFileReader.java:220)
        at
org.apache.lucene.store.InputStream.refill(InputStream.java:158)
        at
org.apache.lucene.store.InputStream.readByte(InputStream.java:43)
        at
org.apache.lucene.store.InputStream.readInt(InputStream.java:73)
        at
org.apache.lucene.store.InputStream.readLong(InputStream.java:96)
        at
org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:59)
        at
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:237)
        at
org.apache.lucene.index.MultiReader.document(MultiReader.java:108)
        at
org.apache.lucene.search.IndexSearcher.doc(IndexSearcher.java:74)
        at org.apache.lucene.search.Hits.doc(Hits.java:101)
        at
deshaw.reccms.indexer.LuceneSearcher.main(LuceneSearcher.java:168)

In both the above cases, I have ascertained that the IndexWriter was
executing iw.addDocument() when the exception was thrown by the process
trying to search the index.

'Lucene in Action' mentions fairly explicitly that searching can be
carried on simultaneously with indexing. So I am not quite sure why this
should happen. 

Thanks,
Venu


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message