lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Huge RAM and small IO bandwidth indexing fast
Date Wed, 10 Sep 2003 13:31:57 GMT

with a bunch of machines with 2GB of RAM but somehow limited IO bandwidth,
due to indexing on a NFS I wonder how I can optimize indexing. Part of the
problem may be that I want to index individual sentences. I have in the
order of
120e6 sentences to index. Looking at the indexing process with 'top' I see
that it
is consuming only 20% of CPU time and is in state 'D'elayed most of the
most probably waiting on IO.

Any ideas how I can tweak the indexing to use more RAM and less IO? I toyed
with IndexWriter.mergeFactor, but have no idea whether to set it to 100 or


COMPUTERBILD 15/03: Premium-e-mail-Dienste im Test
1. GMX TopMail - Platz 1 und Testsieger!
2. GMX ProMail - Platz 2 und Preis-Qualit├Ątssieger!
3. Arcor - 4. - 5. T-Online - 6. - 7. daybyday - 8. e-Post

View raw message