From java-user-return-35245-apmail-lucene-java-user-archive=lucene.apache.org@lucene.apache.org Fri Aug 01 01:13:32 2008 Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 20434 invoked from network); 1 Aug 2008 01:13:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Aug 2008 01:13:32 -0000 Received: (qmail 58003 invoked by uid 500); 1 Aug 2008 01:13:24 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 57966 invoked by uid 500); 1 Aug 2008 01:13:24 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 57955 invoked by uid 99); 1 Aug 2008 01:13:24 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jul 2008 18:13:24 -0700 X-ASF-Spam-Status: No, hits=1.4 required=10.0 tests=SPF_NEUTRAL,WHOIS_MYPRIVREG X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [64.233.184.224] (HELO wr-out-0506.google.com) (64.233.184.224) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Aug 2008 01:12:27 +0000 Received: by wr-out-0506.google.com with SMTP id c30so771438wra.21 for ; Thu, 31 Jul 2008 18:12:51 -0700 (PDT) Received: by 10.90.97.18 with SMTP id u18mr13667586agb.75.1217553171723; Thu, 31 Jul 2008 18:12:51 -0700 (PDT) Received: from ?10.17.4.4? ( [72.93.214.93]) by mx.google.com with ESMTPS id 44sm603411hsa.9.2008.07.31.18.12.50 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 31 Jul 2008 18:12:51 -0700 (PDT) Message-Id: From: Michael McCandless To: java-user@lucene.apache.org In-Reply-To: <18766343.post@talk.nabble.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v926) Subject: Re: FileNotFoundException during indexing Date: Thu, 31 Jul 2008 21:12:50 -0400 References: <18766343.post@talk.nabble.com> X-Mailer: Apple Mail (2.926) X-Virus-Checked: Checked by ClamAV on apache.org Are you only creating one instance of IndexManager and then sharing that instance across all threads? Can you put some logging/printing where you call IndexReader.unLock, to see how often that's happening? That method is dangerous because if you unlock a still-active IndexWriter it leads to exactly this kind of exception. Mike Wojtek212 wrote: > > Hi, > I'm sometimes receiving FileNotFoundExceptions during indexing. > > java.io.FileNotFoundException: /tmp/content/3615.0-3618.0/_3p.fnm > (No such > file or directory) > at > com.test.vcssearch.DefaultServiceIndexer > $2.run(DefaultServiceIndexer.java:245) > at java.lang.Thread.run(Thread.java:595) > Caused by: com.test.search.IndexingException: > java.io.FileNotFoundException: > /tmp/content/3615.0-3618.0/_3p.fnm (No such file or directory) > at > com > .test > .search.impl.lucene.IndexManager.removeDocuments(IndexManager.java: > 293) > at > com > .test > .search.impl.lucene.IndexManager.removeDocuments(IndexManager.java: > 199) > at > com.test.search.impl.lucene.IndexManager.reindex(IndexManager.java: > 250) > at > com.testsearch.impl.lucene.IndexManager.reindex(IndexManager.java:301) > at > com.test.vcssearch.DefaultServiceIndexer > $2.run(DefaultServiceIndexer.java:239) > ... 1 more > Caused by: java.io.FileNotFoundException: /tmp/content/3615.0-3618.0/ > _3p.fnm > (No such file or directory) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.(RandomAccessFile.java:212) > at > org.apache.lucene.store.FSIndexInput > $Descriptor.(FSDirectory.java:497) > at org.apache.lucene.store.FSIndexInput.(FSDirectory.java:522) > at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java: > 434) > at > org > .apache > .lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:204) > at > org > .apache > .lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:169) > at > org > .apache > .lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:153) > at > org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java: > 1601) > at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:900) > at > com > .test > .search.impl.lucene.IndexManager.removeDocuments(IndexManager.java: > 282) > > I have a class with few methods of reindexing, adding and deleting. > Everything is synchronized. Here is the example: > > public class IndexManager { > > private final Object lock = new Object(); > > private final Analyzer analyzer; > > private final FileProxy index; > > IndexManager(FileProxy index, > boolean create) throws IndexingException { > this.analyzer = new TestAnalyzer(); > this.index = index; > if (!create && !index.exists()) { > create = true; > } > > IndexWriter writer = null; > try { > // this checks the directory is unlocked > writer = createIndexWriter(create); > } catch (IOException e) { > throw new IndexingException(e); > } finally { > if (writer != null) { > try { > writer.close(); > } catch (IOException e) { > LOGGER.error("Cannot close index writer", e); > } > } > } > } > > public void reindex(Document[] documents) throws IndexingException { > synchronized (lock) { > removeDocuments(documents); > String[] ids = new String[documents.length]; > for (int i = 0; i < documents.length; i++) { > ids[i] = documents[i].getDocumentID(); > } > addDocuments(documents); > } > } > > private IndexWriter createIndexWriter() throws IOException { > return createIndexWriter(false); > } > > private IndexWriter createIndexWriter(boolean create) throws > IOException > { > > Directory directory = FileProxyDirectory.getDirectory(index, > create); > if (IndexReader.isLocked(directory)) { > IndexReader.unlock(directory); > } > return new IndexWriter(directory,analyzer,create); > } > > public void addDocuments(Document[] documents) > throws IndexingException { > synchronized (lock) { > IndexWriter indexWriter = null; > try { > try { > indexWriter = createIndexWriter(); > for (int i = 0; i < documents.length; i++) { > // add a document ID for future management > if (documents[i].getDocumentID() == null) { > String msg = EXCEPTION_LOCALIZER.format( > "document-id-not-set"); > LOGGER.error(msg); > throw new IndexingException(msg); > } > LuceneDocument luceneDoc = (LuceneDocument) > documents[i]; > > indexWriter.addDocument(luceneDoc.getBackingDocument()); > } > } finally { > if (indexWriter != null) { > try { > indexWriter.close(); > } catch (IOException e) { > LOGGER.error("Cannot close index writer", > e); > } > } > } > } catch (IOException e) { > throw new IndexingException(e); > } > } > } > > public boolean[] removeDocuments(String[] documentIDs) throws > IndexingException { > boolean[] results = new boolean[documentIDs.length]; > > // Batching the removal of a group of documents is more > efficient due > // to the requirement of closing the reader > synchronized (lock) { > try { > IndexWriter indexWriter = null; > try { > indexWriter = createIndexWriter(); > for (int i = 0; i < documentIDs.length; i++) { > Term term = new > Term(SearchConstants.DOCUMENT_ID, > documentIDs[i]); > > indexWriter.deleteDocuments(term); > results[i] = true; > } > indexWriter.optimize(); > } finally { > if (indexWriter != null) { > try { > indexWriter.close(); > } catch (IOException e) { > LOGGER.error("Cannot close index writer", > e); > } > } > } > } catch (IOException e) { > throw new IndexingException(e); > } > } > return results; > } > > } > > This class is used by many threads. Some of them can add > documents, some can delete and reindex. After any operation > IndexWriter is > closed. > I'm using lucene 2.1.0 but even after upgrade to 2.3.2 there is > still the > exception. > I don't use searching during indexing but the exception occurs. Does > anybody > have idea what can be a reason? > -- > View this message in context: http://www.nabble.com/FileNotFoundException-during-indexing-tp18766343p18766343.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-user-help@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org