lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pifpaf...@gmx.de
Subject Huge RAM and small IO bandwidth indexing fast
Date Wed, 10 Sep 2003 13:31:57 GMT
Hi,

with a bunch of machines with 2GB of RAM but somehow limited IO bandwidth,
due to indexing on a NFS I wonder how I can optimize indexing. Part of the
problem may be that I want to index individual sentences. I have in the
order of
120e6 sentences to index. Looking at the indexing process with 'top' I see
that it
is consuming only 20% of CPU time and is in state 'D'elayed most of the
time,
most probably waiting on IO.

Any ideas how I can tweak the indexing to use more RAM and less IO? I toyed
with IndexWriter.mergeFactor, but have no idea whether to set it to 100 or
100000.

  Thanks,
  Harald.

-- 
COMPUTERBILD 15/03: Premium-e-mail-Dienste im Test
--------------------------------------------------
1. GMX TopMail - Platz 1 und Testsieger!
2. GMX ProMail - Platz 2 und Preis-Qualit├Ątssieger!
3. Arcor - 4. web.de - 5. T-Online - 6. freenet.de - 7. daybyday - 8. e-Post


Mime
View raw message