lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amit Kapur" <amitka...@newgen.co.in>
Subject Behaviour of Lucene during Stress/Scalability Test
Date Wed, 02 Apr 2003 09:43:40 GMT
hi everybody

I am trying to index documents using Lucene generating about 30 MB of index (Optimized) which
can be raised to about 100 MB or More ( but that would be on a high end server machine).

Description of Current Case:
#---Each Document has four fields (One Text field, and 3 other Keyword Fields). 
#---The analyzer is based on a StopFilter and a PorterStemFilter.
#---I am using a Compaq PIII, 128 MB RAM, 650 MHz. 
#---mergeFactor is set to 25, and I am optimizing the index after adding about 20 Documents.
#---Using Lucene Release 1.2

Problem Faced
After adding about 4000 Documents generating an index of 30 MB, I initially got an error saying,
**** couldn't rename segments.new to segments **** after which the IndexReader or the IndexWriter
to the current index couldnot be opened.

Then I changed a couple of settings, 
#---mergeFactor=20 and Optimize was called after ever 10 documents.
#---Using Lucene Release 1.3

Problem Faced
After adding about 1500 Documents generating an index of 10 MB, I initially got an error saying,
**** F:\Program Files\OmniDocs Server\ftstest\_3cf.fnm (Too many open files)**** after which
the IndexWriter to the current index couldnot be opened.

Now my requirement needs to have a much much larger index (practically) and I am actually
at the point where these errors are coming unpredictably. 

Please if anyone could guide me on this ASAP.
Thanx in advance

Regards
Amit

PS: I have already read articles in the mail archieve http://www.mail-archive.com/lucene-user@jakarta.apache.org/msg02815.html.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message