lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Ricci <ja...@riccinursery.com>
Subject RE: Too many file?
Date Thu, 23 May 2002 19:47:45 GMT
Peter,

I found the problem. Had nothing to do with optimize. The problem is in the
line:

	Field f = new Field("FIELD_" + (i + 1), Integer.toString(i), true,
true, true);

Should be "FIELD_" + (j + 1); the "i" caused me to have a separate file for
each field. I guess it makes a separate file for each field. Thanks for the
help anyway.

James

-----Original Message-----
From: Peter Carlson [mailto:carlson@bookandhammer.com]
Sent: Thursday, May 23, 2002 3:31 PM
To: Lucene Users List
Subject: Re: Too many file?


This seems a little strange.

I index over 100K documents on a windows machine and I don't get this
problem.

One way to solve it though is to optimize periodically.  Just call

Iw.optimize(); every 50000 records or so. This will consolidate all the
individual files into a single file (or set of files).

Hope that helps.

--Peter


On 5/23/02 12:05 PM, "James Ricci" <james@riccinursery.com> wrote:

> Anyone,
> 
> I am trying to evaluate Lucene for use in our company. I tried the simple
> test below. Before it finished creating the index, I got the exception
> exception below. Examining the directory where the index was created,
there
> were more than 10,000 files created. The error I got was, not
surprisingly,
> too many files open. What have I done wrong here? Should the directory
> contain a single file for each document indexed? Is there a way to limit
> this?
> 
> James
> 
>   static public void createLuceneIndex()
>   {
>       try
>       {
>           IndexWriter iw = new IndexWriter("c:\\temp\\lutest", new
> StandardAnalyzer(), true);
> 
>           for (int i = 0; i < 100000; i++)
>           {
>               Document df = new Document();
> 
>               for (int j = 0; j < 3; j++)
>               {
>                   Field f = new Field("FIELD_" + (i + 1),
> Integer.toString(i), true, true, true);
>                   df.add(f);
>               }
> 
>               iw.addDocument(df);
>               System.out.print('.');
>           }
>       }
>       catch(Throwable t)
>       {
>           t.printStackTrace();
>       }
>   }
> 
> 
> java.io.FileNotFoundException: C:\temp\lutest\_2kk.f27 (Too many
> open files)     
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:98)
> 
> at
> java.io.RandomAccessFile.<init>(RandomAccessFile.java:143)
> at
>
org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java:254
> )     
> at
> org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:262)
> at
> org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:211)
> at
> org.apache.lucene.index.SegmentReader.openNorms(SegmentReader.java:259)
> at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:114)
> at
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:304)
> 
> at
>
org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:283)
> 
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:185)
> at
> ch.abacus.lib.db.net.test.TestMain.createLuceneIndex(TestMain.java:147)
> 
> at ch.abacus.lib.db.net.test.TestMain.main(TestMain.java:61)
> 
> 
> 
> 
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>
> 
> 


--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message