lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jaxtrx" <jax...@xde.net>
Subject RE: Indexing problem
Date Fri, 02 Nov 2001 16:12:07 GMT
Don't know if this helps, but I indexed over 600,000 150k html files on W2K
and linux, then I did 40,000 2mb html files and didn't have any issues.  I
used the demo html indexer class.



-----Original Message-----
From: Daryl Thachuk [mailto:daryl@montagetech.com]
Sent: Friday, November 02, 2001 10:03 AM
To: Lucene Users List
Subject: Re: Indexing problem



A question I'd like answered is, why do I now have to be concerned about
having too many files open when before I didn't? What has changed to
cause this? This sounds like a bug to me.

-d

On Friday, November 2, 2001, at 05:13  AM, Scott Ganyo wrote:

> Yes.  You have too many open files.  There are a few things you can
> try.  1)
> Increase the number of file handles your system has available.  Yes,
> there
> is a setting for this in Windows.   2) Make sure that you have the
> IndexWriter.maxMergeDocs set to Integer.MAX_VALUE (the default).  3) Try
> smaller values for IndexWriter.mergeFactor (default is 10).  4) When all
> else fails, do all your indexing in memory and then write it out to disk
> when you're done.  Doug posted an example of this just a couple days
> ago.
>
------
http://www.montagetech.com


--
To unsubscribe, e-mail:
<mailto:lucene-user-help@jakarta.apache.org>
For additional commands, e-mail:



--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message