Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <29038030.1152815489768.JavaMail.jira@brutus>
Date: Thu, 13 Jul 2006 18:31:29 +0000 (GMT+00:00)
From: "Simon Lorenz (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Created: (LUCENE-628) Intermittent FileNotFoundException for
 .fnm when using rsync
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Intermittent FileNotFoundException for .fnm when using rsync
------------------------------------------------------------

         Key: LUCENE-628
         URL: http://issues.apache.org/jira/browse/LUCENE-628
     Project: Lucene - Java
        Type: Bug

  Components: Search  
    Versions: 1.9    
 Environment: Linux RedHat ES3, Jboss402
    Reporter: Simon Lorenz


We use Lucene 1.9.1 to create and search indexes for web applications. The application runs in Jboss402 on Redhat ES3. A single Master (Writer) Jboss instance creates and writes the indexes using the compound file format , which is optimised after all updates. These index files are replicated every few hours using rsync, to a number of other application servers (Searchers). The rsync job only runs if there are no lucene lock files present on the Writer. The Searcher servers that receive the replicated files, perform only searches on the index. Up to 60 searches may be performed each minute. 

Everything works well most of the time, but we get the following issue on the Searcher servers about 10% of the time. 
Following an rsync replication one or all of the Searcher server throws

IOException caught when creating and IndexSearcher
java.io.FileNotFoundException: /..../_1zm.fnm (No such file or directory)
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
        at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:425)
        at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:434)
        at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:324)
        at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:56)
        at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:144)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
        at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:154)
        at org.apache.lucene.store.Lock$With.run(Lock.java:109)
        at org.apache.lucene.index.IndexReader.open(IndexReader.java:143)  

As we use the compound file format I would not expect .fnm files to be present. When replicating, we do not delete the old .cfs index files as these could still be referenced by old Searcher threads. We do overwrite the segments and deletable files on the Searcher servers. 

My thoughts are: Either we are occasionally overwriting a file at the exact time a new searcher is being created, or the lock files are removed from the Writer server before the compaction process is completed, we then replicate a segments file that still references a ghost .fnm file.

I would greatly appreciate any ideas and suggestions to solve this annoying issue.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org