lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Lorenz (JIRA)" <>
Subject [jira] Created: (LUCENE-628) Intermittent FileNotFoundException for .fnm when using rsync
Date Thu, 13 Jul 2006 18:31:29 GMT
Intermittent FileNotFoundException for .fnm when using rsync

         Key: LUCENE-628
     Project: Lucene - Java
        Type: Bug

  Components: Search  
    Versions: 1.9    
 Environment: Linux RedHat ES3, Jboss402
    Reporter: Simon Lorenz

We use Lucene 1.9.1 to create and search indexes for web applications. The application runs
in Jboss402 on Redhat ES3. A single Master (Writer) Jboss instance creates and writes the
indexes using the compound file format , which is optimised after all updates. These index
files are replicated every few hours using rsync, to a number of other application servers
(Searchers). The rsync job only runs if there are no lucene lock files present on the Writer.
The Searcher servers that receive the replicated files, perform only searches on the index.
Up to 60 searches may be performed each minute. 

Everything works well most of the time, but we get the following issue on the Searcher servers
about 10% of the time. 
Following an rsync replication one or all of the Searcher server throws

IOException caught when creating and IndexSearcher /..../_1zm.fnm (No such file or directory)
        at Method)
        at org.apache.lucene.index.FieldInfos.<init>(
        at org.apache.lucene.index.SegmentReader.initialize(
        at org.apache.lucene.index.SegmentReader.get(
        at org.apache.lucene.index.SegmentReader.get(
        at org.apache.lucene.index.IndexReader$1.doBody(

As we use the compound file format I would not expect .fnm files to be present. When replicating,
we do not delete the old .cfs index files as these could still be referenced by old Searcher
threads. We do overwrite the segments and deletable files on the Searcher servers. 

My thoughts are: Either we are occasionally overwriting a file at the exact time a new searcher
is being created, or the lock files are removed from the Writer server before the compaction
process is completed, we then replicate a segments file that still references a ghost .fnm

I would greatly appreciate any ideas and suggestions to solve this annoying issue.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message