lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject RE: Stress Testing Lucene
Date Wed, 26 Jun 2002 15:33:09 GMT

--- Scott Ganyo <scott.ganyo@eTapestry.com> wrote:
> 1) Are you sure that the index is corrupted?  Maybe the file handles
> just
> haven't been released yet.  Did you try to reboot and try again?

You can also do something like this:

# lsof | wc -l
   8727

# lsof | grep -c java
   5382

# lsof | grep java | head
mozilla-b 8428   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8453   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8454   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8455   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8457   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8471   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
 
> 2) To avoid the too-many files problem: a) increase the system file
> handle
> limits, b) make sure that you reuse IndexReaders as much as you can
> rather
> across requests and client rather than opening and closing them.
> 
> > -----Original Message-----
> > From: Nader S. Henein [mailto:nsh@bayt.net]
> > Sent: Wednesday, June 26, 2002 10:11 AM
> > To: lucene-user@jakarta.apache.org
> > Subject: Stress Testing Lucene
> > Importance: High
> > 
> > 
> > 
> > Hey people,
> > 
> > I'm running a Lucene (v1.2) servlet on resin and I must say 
> > compared to
> > Oracle Intermedia
> > it's working beautifully. BUT today, I started stress testing and I
> > downloaded a program called
> > Web Roller, witch simulates clients, requests , 
> > multi-threading .. the works
> > and I was testing
> > I was doing something like 50 simultaneous requests and I was 
> > repeating that
> > 10 times in a row.
> > 
> > but then something happened and the index got corrupted, 
> > every time I try
> > opening the index
> > with the reader to search or open with the writer to optimize 
> > I get that
> > damned too-many files
> > open error. I can imagine that every application on the market has
> a
> > breaking point and these breaking
> > points have side effects, so is the corruption of the index a 
> > side effect
> > and if so is there a way that
> > I configure my web server to crash before the corruption 
> > occurs, I'd rather
> > re-start the web server and
> > throw some people off wack rather that have to re-build the 
> > index or revert
> > to an older version.
> > 
> > Do you know of any way to safeguard against this ?
> > 
> > General Info:
> > The index is about 45 MB with 60 000 XML files each 
> > containing 18-25 fields.
> > 
> > 
> > Nader S. Henein
> > Bayt.com , Dubai Internet City
> > Tel. +9714 3911900
> > Fax. +9714 3911915
> > GSM. +9715 05659557
> > www.bayt.com
> > 
> > 
> > --
> > To unsubscribe, e-mail:   
> > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail: 
> > <mailto:lucene-user-help@jakarta.apache.org>
> > 
> 


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message