lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marc Dumontier" <>
Subject parallizing index building
Date Fri, 27 Jun 2003 03:57:07 GMT

I'm indexing 500 XML files each ~150Mb on an 8 CPU machine.

I'm wondering what the best strategy for making maximum use of resources is. I have the tweaked
the single process indexer to index 5000 records (not files) in memory before writing out
to disk.

Should i create an IndexThread and share the IndexWriter object across 5 threads..then monitor
when one ends to start another, etc. Or should i create difference indexes then to a series
of merges.

any help would be appreciated,

Marc Dumontier
Bioinformatics Application Developer
Blueprint Initiative
Mount Sinai Hospital

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message