lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: FileNotFoundException in ConcurrentMergeScheduler
Date Tue, 10 Jun 2008 18:54:14 GMT
Hi Paul,

Not sure if this was resolved, but I don't think it was.  Can you try  
reproducing this with setCompoundFile(false)?  That is, turn of  
compound files.  I have an intermittent report of an exception that  
looks eerily similar that I am trying to track down and I am not using  
CFS and the exception is reporting the error in the .fdt (stored  
fields info) file.  Unfortunately, I don't have any more specifics at  
this time as it hasn't been reproduced since, but thought that you  
might be able to get to it quicker since you seem to be able to  
somewhat reproduce it.

Exception is:
java.io.FileNotFoundException: ..../index/_1l5.fdt (No such file or  
directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(Unknown Source)
	at org.apache.lucene.store.FSDirectory$FSIndexInput 
$Descriptor.<init>(FSDirectory.java:506)
	at org.apache.lucene.store.FSDirectory 
$FSIndexInput.<init>(FSDirectory.java:536)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
	at org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:75)
	at  
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:308)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java: 
3263)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
	at org.apache.lucene.index.ConcurrentMergeScheduler 
$MergeThread.run(ConcurrentMergeScheduler.java:240)

Of course, this may just be a red herring, since it is a background  
thread running and it may just be a timing thing, but it might be  
interesting if it consistently occurred in the same spot.


Thanks,
Grant

On May 29, 2008, at 7:43 PM, Paul J. Lucas wrote:

> I occasionally get a FileNotFoundException like:
>
> Exception in thread "Thread-44" org.apache.lucene.index.MergePolicy 
> $MergeException: java.io.FileNotFoundException: /Stuff/Caches/ 
> AuroraSupport/IM_IndexCache/INDEX/_27.cfs (No such file or directory)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler 
> $MergeThread.run(ConcurrentMergeScheduler.java:271)
> Caused by: java.io.FileNotFoundException: /Stuff/Caches/ 
> AuroraSupport/IM_IndexCache/INDEX/_27.cfs (No such file or directory)
> 	at java.io.RandomAccessFile.open(Native Method)
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
> 	at org.apache.lucene.store.FSDirectory$FSIndexInput 
> $Descriptor.<init>(FSDirectory.java:506)
> 	at org.apache.lucene.store.FSDirectory 
> $FSIndexInput.<init>(FSDirectory.java:536)
> 	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java: 
> 445)
> 	at  
> org 
> .apache 
> .lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> 	at  
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java: 
> 277)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
> 	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java: 
> 3263)
> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
> 	at org.apache.lucene.index.ConcurrentMergeScheduler 
> $MergeThread.run(ConcurrentMergeScheduler.java:240)
>
> I'm currently using the 2.3.2 version of Lucene.  The exception  
> frequency has decreased since upgrading from 2.3.1.
>
> My code runs in a server that monitors the filesystem.  Whenever the  
> contents of a directory change, my code unindexes all the files in  
> the changed directory, then reindexes all the files in the directory.
>
> I'm not doing anything complicated in my code.  To create/open the  
> index, I do:
>
> 	INDEX_DIR = new File( INDEX_CACHE_DIR, "INDEX" );
> 	INDEX = FSDirectory.getDirectory( INDEX_DIR );
> 	if ( IndexReader.isLocked( INDEX ) )
> 	    IndexReader.unlock( INDEX );
>
> The isLocked()/unlock() is because sometimes the server process gets  
> killed and leaves teh indexed locked.
>
> I have a thread than handles the unindexing/reindexing.  It gets  
> changed from a BlockingQueue.  My unindex code is like:
>
> 	IndexWriter writer = new IndexWriter( INDEX, INDEX_ANALYZER, false );
> 	final Term t = new Term( DIR_FIELD, dir.getAbsolutePath() );
> 	writer.deleteDocuments( t );
> 	writer.close();
>
> My indexing code is like:
>
> 	IndexWriter writer = new IndexWriter( INDEX, INDEX_ANALYZER );
> 	writer.setMergeFactor( INDEX_MERGE_FACTOR );
> 	Document doc = new Document();
> 	// ... add fields to doc ...
> 	writer.addDocument( doc );
> 	writer.close();
>
> While that thread is executing, other threads can search the index  
> by doing:
>
> 	IndexSearcher searcher = new IndexSearcher( INDEX );
> 	// ... prepare Query and Sort ...
> 	Hits hits = searcher.search( query, sort );
> 	Iterator hitIterator  hits.iterator();
> 	// ... iterate over hitIterator ...
> 	searcher.close();
>
> Any ideas?  Is this a bug in Lucene?
>
> - Paul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--------------------------
Grant Ingersoll
http://www.lucidimagination.com

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ








---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message