lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Re: Does more memory help Lucene?
Date Mon, 12 Jun 2006 13:36:45 GMT

Look up one of my Lucene articles, where I talk about this.  You may also want
to tell Lucene to merge segments on disk less frequently, which is what mergeFactor does.


----- Original Message ----
From: Nadav Har'El <>
Sent: Monday, June 12, 2006 8:13:10 AM
Subject: Does more memory help Lucene?


I am trying to index a huge collection of documents - several hundreds of
Needless to say, I'm trying to squeeze every ounce of performance from my
machine, to get this indexing done in a sensible amount of time.

Making use of the fact that my machine has two CPUs was easy: I just run
two indexing threads, creating two separate indices which can later be
merged, or searched with a MultiSearch.

What I couldn't figure out how to use, however, was the abundant memory (2
GB) that this machine has.

I tried playing with IndexWriter.setMaxBufferedDocs(), and noticed that
there is no speed gain after I set it to 1000, at which point the running
Lucene takes up just 70 MB of memory, or 140 MB for the two threads.

Is there a way for Lucene to make use of the much larger memory I have, to
speed up the indexing process? Does having a huge memory somehow improve
the speed of huge merges, for example?


Nadav Har'El

To unsubscribe, e-mail:
For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message