lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Large Indexes
Date Mon, 14 Nov 2005 04:40:55 GMT
Largest index?  Who knows! :)
Lucene's internal limit is the size of the doc Id (max Integer).

People typically roll their indices when they reach a certain size, but
if you don't need your queries to be fast and always need all the data,
then this may not make sense for you (well, it still may, as you can
distribute such indices over multiple machines and query them in
parallel, aggregating the results).

What to expect?  Slower queries, longer optimization, more disk space,
higher memory usage (larget .tii read in memory), the usual stuff.

Otis

--- "Friedland, Zachary (EDS - Strategy)" <zachary_friedland@ml.com>
wrote:

> What is the largest lucene index that has been built?  We're looking
> to
> build a sort of data warehouse that will hold transaction log files
> as
> long as possible.  This index would grow at the rate of 10 million
> documents per month indefinitely.  Is there a limit where lucene will
> fail?  What should we expect to happen as this index grows?
> 
> Thank you in advance,
> Zach
> --------------------------------------------------------
> 
> If you are not an intended recipient of this e-mail, please notify
> the sender, delete it and do not read, act upon, print, disclose,
> copy, retain or redistribute it. Click here for important additional
> terms relating to this e-mail.     http://www.ml.com/email_terms/
> --------------------------------------------------------
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message