lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Øie <k...@gan.no>
Subject RE: optimizing index - too many open files
Date Fri, 01 Mar 2002 15:23:47 GMT
I have to index 1650mb of documents, and eventually i will get "out of
memory" with a RAMDir and get "too many open files" with a FSdir, so to get
around this i am indexing 100 documents at a time in a RAMDir, then merge
this RAMDir into a FSDir before i index the next set of 100 files. This made
me work around both "out of memory" and "too many files" exceptions...

mvh karl øie



-----Original Message-----
From: Paul Friedman [mailto:paulsanfordfriedman@yahoo.com]
Sent: 28. februar 2002 21:38
To: Lucene Users List
Subject: Re: optimizing index - too many open files


Sorry to bother y'all again.
Found an answer in the archives under the Thread "Indexing problem".

About to try using RAMDirectory first.

pax et bonum. p.

----- Original Message -----
From: "Paul Friedman" <paulsanfordfriedman@yahoo.com>
To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Sent: Thursday, February 28, 2002 1:23 PM
Subject: optimizing index - too many open files


> Hello all,
>
> I am running into an error:
> java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
> files )
> after my class calls IndexWriter.optimize().
>
> Does anybody know what causes this error?
> Any help is appreciated.
>
> ( By the way, the site that I am indexing is huge.
>   I have a crawler run through the site calling many .jsps, .pdfs, and
.html
> docs.
>   It ran fine two days ago after indexing 3700+ pages. )
>
> Could the index be too large for Lucene to handle?
>
> The error:
> java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
> files )
>     at java.io.RandomAccessFile.open( Native Method )
>     at java.io.RandomAccessFile.<init>
>     at java.io.RandomAccessFile.<init>
>     at org.apache.lucene.store.FSInputStream$Descriptor.<init>
>     at org.apache.lucene.store.FSInputStream.<init>
>     at org.apache.lucene.store.FSDirectory.openFile
>     at org.apache.lucene.index.TermInfosReader.readIndex
>     at org.apache.lucene.index.TermInfosReader.<init>
>     at org.apache.lucene.index.SegmentReader.<init>
>     at org.apache.lucene.index.IndexWriter.mergeSegments
>     at org.apache.lucene.index.IndexWriter.optimize
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com


--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>


--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message