Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 20271 invoked from network); 2 Sep 2005 04:33:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 2 Sep 2005 04:33:37 -0000 Received: (qmail 48428 invoked by uid 500); 2 Sep 2005 04:33:31 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 48407 invoked by uid 500); 2 Sep 2005 04:33:31 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 48391 invoked by uid 99); 2 Sep 2005 04:33:31 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Sep 2005 21:33:31 -0700 X-ASF-Spam-Status: No, hits=0.4 required=10.0 tests=DNS_FROM_RFC_ABUSE X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: local policy) Received: from [68.142.200.42] (HELO web31109.mail.mud.yahoo.com) (68.142.200.42) by apache.org (qpsmtpd/0.29) with SMTP; Thu, 01 Sep 2005 21:33:46 -0700 Received: (qmail 83691 invoked by uid 60001); 2 Sep 2005 04:33:29 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=D25GmeIEaMhZlXWk7wAYMsgrwV7s5Bp0O9ZrcElTjV9RvGLHg/ZB20OAgUPoEbKNx70YSUUz8+OzrOGVIUDY0qwqWFQh3i/oWVOjsu4JJzPmUFA2tCDedFXg2fFnaLgUkR+7mqStfthZA/hN1i6ItJ5lKVCVb3jkU77cGBIHeSY= ; Message-ID: <20050902043329.83689.qmail@web31109.mail.mud.yahoo.com> Received: from [69.201.142.21] by web31109.mail.mud.yahoo.com via HTTP; Thu, 01 Sep 2005 21:33:29 PDT Date: Thu, 1 Sep 2005 21:33:29 -0700 (PDT) From: Otis Gospodnetic Subject: Re: Too many open files when doing performance testing To: java-user@lucene.apache.org Cc: Amaladason Thadeuse , Vicky Tsai In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N Use that compound index format and use a single IndexSearcher instance. Otis --- Akhil Jain wrote: > > > > > > Actually, we might not be able to change the setting as we are using > a > shared server and are not allowed to change system settings. What > other > things can we look at to get this fixed? > > > Akhil Jain > > > > > > > Otis Gospodnetic > > > @yahoo.com> > To > java-user@lucene.apache.org > > 09/01/2005 09:07 > cc > PM > > > Subject > Re: Too many open files when > doing > Please respond to performance testing > > java-user > > > > > > > > > > > > > > > > Hi, > > 2000 doesn't sound very high. I've used much higher values. Because > you have so many fields and files, you may want to increase it. If > you > have Lucene in Action, we have a formula in there for calculating the > number of open files used (section 2.7, according to > http://www.lucenebook.com/search?query=open+files+maximum) > > You said you got this error during your performance testing. Are you > searching a single index and using a single IndexSearcher? Use a > single IndexSearcher. > > Otis > > > --- "jaina (sent by Nabble.com)" wrote: > > > > > Thanks Otis for your questions. Here are the answers... Please let > > me know your thoughts.... > > > > We do optimize() after indexWrite. > > The maximum open file allowed is 2,000. Is this normal, too small? > > What is your recommended number ? We have 57 index files, how many > > files that Lucene index will actually opens per session? > > We use default mergeFactor 10, default merge Docs 1000. > > We have about 18,000 documents, 23 indexed fields. > > We are not using compound index because, but we are considering it. > > Following is a snapshot of our typical index files: > > > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f1 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f10 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f11 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f12 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f13 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f14 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f15 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f16 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f17 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f18 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f19 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f2 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f20 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f21 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f22 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f23 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f24 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f25 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f26 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f27 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f28 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f29 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f3 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f30 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f31 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f32 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f33 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f34 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f35 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f36 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f37 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f38 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f39 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f4 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f40 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f41 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f42 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f43 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f44 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f45 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f46 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f47 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f48 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f49 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f5 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f50 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f51 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f6 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f7 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f8 > > -rw-rw---- 1 skpmq p/gsa_cd 18719 Sep 01 09:18 _bgw.f9 > > -rw-rw---- 1 skpmq p/gsa_cd 18624312 Sep 01 09:18 _bgw.fdt > > -rw-rw---- 1 skpmq p/gsa_cd 149752 Sep 01 09:18 _bgw.fdx > > -rw-rw---- 1 skpmq p/gsa_cd 579 Sep 01 09:18 _bgw.fnm > > -rw-rw---- 1 skpmq p/gsa_cd 23085952 Sep 01 09:18 _bgw.frq > > -rw-rw---- 1 skpmq p/gsa_cd 42672641 Sep 01 09:18 _bgw.prx > > -rw-rw---- 1 skpmq p/gsa_cd 145555 Sep 01 09:18 _bgw.tii > > -rw-rw---- 1 skpmq p/gsa_cd 11866281 Sep 01 09:18 _bgw.tis > > -rw-rw---- 1 skpmq p/gsa_cd 4 Sep 01 09:18 deletable > > -rw-rw---- 1 skpmq p/gsa_cd 29 Sep 01 09:18 segments > > -- > > Sent from the Lucene - Java Users forum at Nabble.com: > > > http://www.nabble.com/Too-many-open-files-when-doing-performance-testing-t272369.html#a766958 > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-user-help@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org