lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Query:Adding all docs at once or creating smaller indexes and merge
Date Fri, 05 Jun 2009 11:01:37 GMT
Hi


My guess is that one big index would be more efficient since the total
IO read and write load would be less.  The big reason for creating
smaller intermediate indexes is that you could spread their creation
over multiple jobs/disks/servers.

There is lots of good advice in
http://wiki.apache.org/lucene-java/ImproveIndexingSpeed

As to a performance penalty, there are too many variables to hazard a
guess.  Try it out in your application on your hardware.


--
Ian.


On Thu, Jun 4, 2009 at 10:10 PM, Tarandeep Singh<tarandeep@gmail.com> wrote:
> Hi,
>
> From efficiency point of view, what will be more efficient-
>
> Creating a single big index (big enough for one machine) by adding all
> documents in it at once
>
> or
>
> Creating smaller indexes and then merge them to make one bigger index?
>
> And if there is a performance penalty, then any rough estimate how much?
>
> Thanks,
> Tarandeep
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message