lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Krah <krah...@gmail.com>
Subject Re: “Too many open files" error hit when I run my Lucene 3.0.3 application on Java 8
Date Wed, 22 Feb 2017 16:30:33 GMT
What does lsof tell - which handles are open and which are unexpected or
suspicious?

Am 22.02.2017 5:27 nachm. schrieb "Leonid Bolshinsky" <leonidbol@gmail.com>:

> No code changes between running in Java 6 and Java 8. The same code still
> runs successfully with Java 6 while is failing with Java 8.
>
> On Feb 22, 2017 5:52 PM, "Frederik Van Hoyweghen" <
> frederik.vanhoyweghen@chapoo.com> wrote:
>
> Did you make any changes to your code between running on java 6 and 8?
> The java file API has changed considerably since java 8.
>
> If you -have- rewritten some of the file handling code in your indexing
> process, make sure to explicitly close the streams you create, or use the
> (since java 7) try-with-resources construct.
>
>
> On 22/02/2017 16:18, Leonid Bolshinsky wrote:
>
> > I have a search engine based on Lucene 3.0.3 and I can't change the
> Lucene
> > version for reasons that are out of scope of this question. Now I have a
> > requirement to move from Java 6 to Java 8, however when I run the
> indexing
> > using Java 8 JVM, I hit "Too many open files issue" as below:
> >
> > java.io.FileNotFoundException: /myIndex/_27c.fdx (Too many open files)
> >      at java.io.RandomAccessFile.open0(Native Method)
> >      at java.io.RandomAccessFile.open(RandomAccessFile.java:333)
> >      at java.io.RandomAccessFile.<init>(RandomAccessFile.java:257)
> >      at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutpu
> > t.<init>(SimpleFSDirectory.java:180)
> >      at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDir
> > ectory.java:85)
> >      at org.apache.lucene.index.FieldsWriter.<init>(
> FieldsWriter.java:86)
> >      at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(
> > StoredFieldsWriter.java:66)
> >      at org.apache.lucene.index.StoredFieldsWriter.finishDocument(St
> > oredFieldsWriter.java:144)
> >      at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(
> > StoredFieldsWriter.java:192)
> >      at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocum
> > ent(DocumentsWriter.java:1559)
> >      at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(Docume
> > ntsWriter.java:1578)
> >      at org.apache.lucene.index.DocumentsWriter.finishDocument(Docum
> > entsWriter.java:1155)
> >      at org.apache.lucene.index.DocumentsWriter.updateDocument(Docum
> > entsWriter.java:832)
> >      at org.apache.lucene.index.DocumentsWriter.updateDocument(Docum
> > entsWriter.java:807)
> >      at org.apache.lucene.index.IndexWriter.updateDocument(IndexWrit
> > er.java:2155)
> >
> > The ulimit on my system is 80.000 files and when I run lsof, I get that
> the
> > open files belong to the index and they are marked as (deleted)
> segments. I
> > am aware that there are tons of questions, answers and blogs about the
> > Lucene open files issue, however:
> >
> >     - I've already tried whatever is recommended (review the code looking
> >
> >     for open IndexWriters, increase ulimit, enable compound mode,
> decrease
> > the
> >     number of segments from 10 to 4, review the code for open files)
> > without
> >     success and
> >     - The issue didn't appear with Java 6 so I am guessing that the
> problem
> >
> >     is in how Java 8 handles files.
> >
> > Anybody faced this issue with Java 8 before? Any additional idea about
> how
> > to further troubleshoot this issue or what could be the cause?
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message