lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Serebrennikov <dmit...@earthlink.net>
Subject Re: Too many open files
Date Thu, 04 Mar 2004 22:32:53 GMT
You can tell how many file handles you are allowed to open by your OS 
(looks like some flavor of Unix from the paths that you have included).  
One way to reduce the number of files Lucene opens is to use compound 
indexes (where each index segment uses a single file). Look for this 
flag on IndexWriter object. You will have to optimize existing indexes 
to convert them.

If you need to know how to tell how many files you are allowed to open 
on Unix, the command is ulimit. Try ulimit -Ha and look for 
"nofiles(descriptors)" - this works on Solaris.
There are usually two limits for each variable - hard and soft. Soft 
limit you can control with the ulimit command before you start your 
process. You can rase it as far as the hard limit. The hard limit is 
given in the /etc/system file (again for Solaris) and is generally not 
changeable except by root. You will find many references on the web that 
say not to increase this parameter in /etc/system beyound 2000 or so, 
but in reality any significant J2EE installation on Solaris will have 
8000 to 9000 setting.

Good luck.
Dmitry.


karl wettin wrote:

>I'm getting this exception, and I can't explain it. It only occurs when
>calling my method that retrieves the content from my index. I get the
>same exception in 1.2 as in 1.3-final, been searching the web all over
>the place but can't find anything else than the same problem described,
>no solution.
>
>
>
>It takes some 50-100 calls to the method until the exception in thrown. 
>Don't I close the IndexReader correct? Is there anything else I need to
>close? 
>	
>Greatful for any hints.
>
>
>----> exception snippet
>
>java.io.FileNotFoundException: /home/kalle/projects/snigel/egdelon_data/lucene/_2ur.fdx
(Too many open files)
>        at java.io.RandomAccessFile.open(Native Method)
>        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:200)
>        at org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java:389)
>        at org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:418)
>        at org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:291)
>        at org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:80)
>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:139)
>        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:120)
>        at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:122)
>        at org.apache.lucene.store.Lock$With.run(Lock.java:148)
>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:99)
>        at org.apache.lucene.search.IndexSearcher.<init>(IndexSearcher.java:75)
>        at se.snigel.egdelon.document.indexer.LuceneIndexer.getSegmentContents(LuceneIndexer.java:48)
>
>row 48 >> s = new IndexSearcher(Egdelon.getSetting("lucene.path"));
>
>----> full method 
>
>    public String getSegmentContents(Segment segment)
>                              throws IOException {
>        File path = new File(Egdelon.getSetting("lucene.path"));
>
>        if (!IndexReader.indexExists(path))
>            return null;
>
>        Searcher s = null;
>
>        try {
>            Analyzer a = new StandardAnalyzer();
>            Query q = QueryParser.parse(String.valueOf(segment.getController().getPrimaryKey().getPK()),
>                                        "segment.pk", a);
>            s = new IndexSearcher(Egdelon.getSetting("lucene.path"));  
>
>            Hits hits = s.search(q);
>
>            if (hits.length() == 1)
>                return hits.doc(0).getField("contents").stringValue();
>
>            s.close();
>
>            return null;
>        } catch (ParseException e) {
>            e.printStackTrace();
>
>            if (s != null)
>                s.close();
>
>            return null;
>        }
>    }
>
>  
>



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message