lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Huge RAM and small IO bandwidth indexing fast
Date Wed, 10 Sep 2003 14:28:46 GMT
I included some information about improving indexing performance in my
second Lucene article on Onjava.com.  The article was published a few
months ago, so you'll have to enter Lucene in the search box on
Onjava.com site.

Otis

--- pifpafpuf@gmx.de wrote:
> Hi,
> 
> with a bunch of machines with 2GB of RAM but somehow limited IO
> bandwidth,
> due to indexing on a NFS I wonder how I can optimize indexing. Part
> of the
> problem may be that I want to index individual sentences. I have in
> the
> order of
> 120e6 sentences to index. Looking at the indexing process with 'top'
> I see
> that it
> is consuming only 20% of CPU time and is in state 'D'elayed most of
> the
> time,
> most probably waiting on IO.
> 
> Any ideas how I can tweak the indexing to use more RAM and less IO? I
> toyed
> with IndexWriter.mergeFactor, but have no idea whether to set it to
> 100 or
> 100000.
> 
>   Thanks,
>   Harald.
> 
> -- 
> COMPUTERBILD 15/03: Premium-e-mail-Dienste im Test
> --------------------------------------------------
> 1. GMX TopMail - Platz 1 und Testsieger!
> 2. GMX ProMail - Platz 2 und Preis-Qualitätssieger!
> 3. Arcor - 4. web.de - 5. T-Online - 6. freenet.de - 7. daybyday - 8.
> e-Post
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 


__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com

Mime
View raw message