lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Jain <ja...@us.ibm.com>
Subject Re: Too many open files when doing performance testing
Date Fri, 02 Sep 2005 04:22:23 GMT





Actually, we might not be able to change the setting as we are using a
shared server and are not allowed to change system settings.   What other
things can we look at to get this fixed?


Akhil Jain




                                                                           
             Otis Gospodnetic                                              
             <otis_gospodnetic                                             
             @yahoo.com>                                                To 
                                       java-user@lucene.apache.org         
             09/01/2005 09:07                                           cc 
             PM                                                            
                                                                   Subject 
                                       Re: Too many open files when doing  
             Please respond to         performance testing                 
                 java-user                                                 
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




Hi,

2000 doesn't sound very high.  I've used much higher values.  Because
you have so many fields and files, you may want to increase it.  If you
have Lucene in Action, we have a formula in there for calculating the
number of open files used (section 2.7, according to
http://www.lucenebook.com/search?query=open+files+maximum)

You said you got this error during your performance testing.  Are you
searching a single index and using a single IndexSearcher?  Use a
single IndexSearcher.

Otis


--- "jaina (sent by Nabble.com)" <lists@nabble.com> wrote:

>
> Thanks Otis for your questions.  Here are the answers... Please let
> me know your thoughts....
>
> We do optimize() after indexWrite.
> The maximum open file allowed is 2,000. Is this normal, too small?
> What is your recommended number ?  We have 57 index files, how many
> files that Lucene index will actually opens per session?
> We use default mergeFactor 10, default merge Docs 1000.
> We have about 18,000 documents, 23 indexed fields.
> We are not using compound index because, but we are considering it.
> Following is a snapshot of our typical index files:
>
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f1
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f10
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f11
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f12
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f13
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f14
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f15
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f16
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f17
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f18
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f19
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f2
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f20
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f21
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f22
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f23
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f24
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f25
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f26
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f27
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f28
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f29
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f3
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f30
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f31
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f32
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f33
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f34
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f35
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f36
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f37
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f38
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f39
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f4
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f40
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f41
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f42
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f43
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f44
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f45
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f46
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f47
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f48
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f49
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f5
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f50
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f51
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f6
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f7
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f8
> -rw-rw----   1 skpmq    p/gsa_cd      18719 Sep 01 09:18 _bgw.f9
> -rw-rw----   1 skpmq    p/gsa_cd   18624312 Sep 01 09:18 _bgw.fdt
> -rw-rw----   1 skpmq    p/gsa_cd     149752 Sep 01 09:18 _bgw.fdx
> -rw-rw----   1 skpmq    p/gsa_cd        579 Sep 01 09:18 _bgw.fnm
> -rw-rw----   1 skpmq    p/gsa_cd   23085952 Sep 01 09:18 _bgw.frq
> -rw-rw----   1 skpmq    p/gsa_cd   42672641 Sep 01 09:18 _bgw.prx
> -rw-rw----   1 skpmq    p/gsa_cd     145555 Sep 01 09:18 _bgw.tii
> -rw-rw----   1 skpmq    p/gsa_cd   11866281 Sep 01 09:18 _bgw.tis
> -rw-rw----   1 skpmq    p/gsa_cd          4 Sep 01 09:18 deletable
> -rw-rw----   1 skpmq    p/gsa_cd         29 Sep 01 09:18 segments
> --
> Sent from the Lucene - Java Users forum at Nabble.com:
>
http://www.nabble.com/Too-many-open-files-when-doing-performance-testing-t272369.html#a766958

>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message