lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nader S. Henein" <...@bayt.net>
Subject RE: Stress Testing Lucene
Date Thu, 27 Jun 2002 04:51:09 GMT
sorry .. but still the same problem, I've saved the index in a seperate
direcory
and I've re-indexed overnight so, testing (witch is currently underway) on
the
system can resume. Like I said in my previous email , worst case scenarios
and the index
is corrupted any ideas as to why, I'll gladly go into the source but some
guidance as
to a starting point would be nice


-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Wednesday, June 26, 2002 7:33 PM
To: Lucene Users List
Subject: RE: Stress Testing Lucene



--- Scott Ganyo <scott.ganyo@eTapestry.com> wrote:
> 1) Are you sure that the index is corrupted?  Maybe the file handles
> just
> haven't been released yet.  Did you try to reboot and try again?

You can also do something like this:

# lsof | wc -l
   8727

# lsof | grep -c java
   5382

# lsof | grep java | head
mozilla-b 8428   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8453   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8454   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8455   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8457   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so
mozilla-b 8471   otis  mem    REG        3,5  1242726   1287892
/usr/local/.version/IBMJava2-13/jre/bin/libjavaplugin_oji.so

> 2) To avoid the too-many files problem: a) increase the system file
> handle
> limits, b) make sure that you reuse IndexReaders as much as you can
> rather
> across requests and client rather than opening and closing them.
>
> > -----Original Message-----
> > From: Nader S. Henein [mailto:nsh@bayt.net]
> > Sent: Wednesday, June 26, 2002 10:11 AM
> > To: lucene-user@jakarta.apache.org
> > Subject: Stress Testing Lucene
> > Importance: High
> >
> >
> >
> > Hey people,
> >
> > I'm running a Lucene (v1.2) servlet on resin and I must say
> > compared to
> > Oracle Intermedia
> > it's working beautifully. BUT today, I started stress testing and I
> > downloaded a program called
> > Web Roller, witch simulates clients, requests ,
> > multi-threading .. the works
> > and I was testing
> > I was doing something like 50 simultaneous requests and I was
> > repeating that
> > 10 times in a row.
> >
> > but then something happened and the index got corrupted,
> > every time I try
> > opening the index
> > with the reader to search or open with the writer to optimize
> > I get that
> > damned too-many files
> > open error. I can imagine that every application on the market has
> a
> > breaking point and these breaking
> > points have side effects, so is the corruption of the index a
> > side effect
> > and if so is there a way that
> > I configure my web server to crash before the corruption
> > occurs, I'd rather
> > re-start the web server and
> > throw some people off wack rather that have to re-build the
> > index or revert
> > to an older version.
> >
> > Do you know of any way to safeguard against this ?
> >
> > General Info:
> > The index is about 45 MB with 60 000 XML files each
> > containing 18-25 fields.
> >
> >
> > Nader S. Henein
> > Bayt.com , Dubai Internet City
> > Tel. +9714 3911900
> > Fax. +9714 3911915
> > GSM. +9715 05659557
> > www.bayt.com
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> > For additional commands, e-mail:
> > <mailto:lucene-user-help@jakarta.apache.org>
> >
>


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

--
To unsubscribe, e-mail:
<mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>




--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message