lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Friedman <pfried...@macromedia.com>
Subject RE: Indexing problem
Date Fri, 02 Nov 2001 14:12:58 GMT
Where can I get Doug's example of indexing in memory and then writing it out
to disk? I just recently subscribed to this list and I can't find it in the
archive.  

Thanks.
Paul

-----Original Message-----
From: Scott Ganyo [mailto:scott.ganyo@eTapestry.com]
Sent: Friday, November 02, 2001 7:14 AM
To: 'Lucene Users List'
Subject: RE: Indexing problem


Yes.  You have too many open files.  There are a few things you can try.  1)
Increase the number of file handles your system has available.  Yes, there
is a setting for this in Windows.   2) Make sure that you have the
IndexWriter.maxMergeDocs set to Integer.MAX_VALUE (the default).  3) Try
smaller values for IndexWriter.mergeFactor (default is 10).  4) When all
else fails, do all your indexing in memory and then write it out to disk
when you're done.  Doug posted an example of this just a couple days ago.

Scott

> -----Original Message-----
> From: Daryl Thachuk [mailto:daryl@montagetech.com]
> Sent: Thursday, November 01, 2001 7:46 PM
> To: lucene-user@jakarta.apache.org
> Subject: Indexing problem
> 
> 
> Hi
> 
> Since upgrading to 1.2  we've started getting the following 
> error when 
> creating an index in a directory with a large amount of 
> files. Previous 
> versions of Lucene were quite happy to index this directory.
> 
> Any thoughts as to the cause?
> 
> -d
> 
> java.io.FileNotFoundException: 
> /private/Network/Servers/montage/Volumes/Disk2/Users/daryl/Lib
> rary/Index/index5.
> mtlibx/_n8.f3 (Too many open files)
> 	at java.io.RandomAccessFile.open(Native Method)
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
> 	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:143)
> 	at 
> org.apache.lucene.store.FSInputStream$Descriptor.<init>(Unknown 
> Source)
> 	at org.apache.lucene.store.FSInputStream.<init>(Unknown Source)
> 	at org.apache.lucene.store.FSDirectory.openFile(Unknown Source)
> 	at 
> org.apache.lucene.index.SegmentReader.openNorms(Unknown Source)
> 	at org.apache.lucene.index.SegmentReader.<init>(Unknown Source)
> 	at 
> org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source)
> 	at 
> org.apache.lucene.index.IndexWriter.mergeSegments(Unknown Source)
> 	at 
> org.apache.lucene.index.IndexWriter.maybeMergeSegments(Unknown 
> Source)
> 	at 
> org.apache.lucene.index.IndexWriter.addDocument(Unknown Source)
> 	at IndexCreator.indexDocs(IndexCreator.java:75)
> 	at IndexCreator.indexDocs(IndexCreator.java:67)
> 	at IndexCreator.indexDocs(IndexCreator.java:67)
> 	at IndexCreator.indexDocs(IndexCreator.java:67)
> 	at IndexCreator.indexDocs(IndexCreator.java:67)
> 	at IndexCreator.indexDocs(IndexCreator.java:67)
> 	at IndexCreator.createIndex(IndexCreator.java:44)
> 
> ------
> Daryl Thachuk		daryl@montagetech.com
> Montage Technologies Inc.
> http://www.montagetech.com
> 
> 
> --
> To unsubscribe, e-mail:   
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message