lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik N S" <kart...@controlnet.co.in>
Subject RE: Too many Open Files + lucene 1.4.1 + Linux O/s
Date Wed, 13 Oct 2004 09:53:32 GMT
Hi


Apologies for  the Long wait..........


   My Linux system on ulimit -a  respresent


core file size                   (blocks, -c) 0
data seg size                  (kbytes, -d) unlimited
file size                            (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                        (-n) 1024
pipe size                          (512 bytes, -p) 8
stack size                       (kbytes, -s) 8192
cpu time                         (seconds, -t) unlimited
max user processes    (-u) 1983
virtual memory             (kbytes, -v) unlimited


The Problem of  "Too many Open Files"  happens on every 2nd Search  being
done

I think as u say  "open files                    (-n) 1024"   should be
increased...........


More Advises  is Accepted  greatefully....

Thx in advance





-----Original Message-----
From: Dmitry Serebrennikov [mailto:dmitrys@earthlink.net]
Sent: Sunday, October 03, 2004 5:08 AM
To: Lucene Users List
Subject: Re: Too many Open Files + lucene 1.4.1 + Linux O/s


Karthik N S wrote:

>Hi Luceners,
>
>
>Apologies.........
>
>
>Other day was Trying to Search using the "Luceneweb"  version
>with Lucene1-4-1.zip  and   O/s = Linux, J2SDK version "1.4.2_03-b02"
>
>With Roughly around  500 Documents (715116 kb )  Indexed  using
>Lucene1.4-final.jar and  writer.setUseCompoundFile(true);
>
>
Here are a couple of possibilities:
- the setUseCompoundFile(true) will only apply to indexes created (or
optimized) after the option is set.
  All pre-existing indexes will still be in the multi-file format.
- number of documents does not directly impact the number of files
needed by Lucene. If the index is
  really in a compound file format (see above), and is optimized, you
will need a fixed number of file handles.
  Even if the index is in a multi-file format, the number of files
needed depends on the number of indexed *fields* in the index (not
documents).
- do you get the error on the first and every search or only once in a
while? Perhaps where there are lots of
  concurrent users? Perhaps after you've done X searchers?
- check your OS-level setting for the number of open files. This is
shell/system-dependent somewhat, but
   "ulimit -a" should get you started. The number of open files should
be large enough to allow for all files
   and sockets that your application needs to open. In a typical
server-side Java app setting this value should
   be around 8000. Defaults are much smaller, so unless you have changed
this, this may be the answer.
- look into "lsof" utility. It can display all file handles in use by a
given process. This is a good tool to
  troubleshoot "too many open files" issues.

Good luck.
Dmitry.


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message