lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu" <chris...@gmail.com>
Subject Re: Lucene index sizes and performance
Date Sun, 08 Jul 2007 03:42:04 GMT
Not really suggestion but some points to consider.
(a) Greatly depending on your hardware, especially harddrive speed.
(b) Do you do SortBy? Each SortBy field will need an array in memory.
If no sortBy, reserve memory for about 10~15% of index size will be enough.
(c) Maybe try to split by index content category first, much easier.

-- 
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes

On 7/7/07, Chun Wei Ho <cwho.work@gmail.com> wrote:
> We are currently running a search service with a single Lucene index
> of about 10 GB. We would like to find out:
>
> (a) What is the usual index size of everyone else? How large have
> Lucene index gone in prodution environments, and is there a sort of a
> optimal size that Lucene indexes should be?
>
> (b) With a index size of 10GB, how much memory would you recommend a
> dual 3GHz machine serving searches on it to have. We currently have
> 4GB RAM and are thinking of adding more for faster searches?
>
> Is there a ballpark figure or guide that we can adhere to - so we
> might add more RAM depending on the rate of index growth.
>
>
> (c) We're considered the possibility of splitting our large index into
> several smaller ones based on discussions in previous threads.
>
> Did anyone do so here and how did you manage it - splitting by logical
> category, or splitting by time (so perhaps a index that holds 2 months
> worth of documents might be split into 8 indexes of 1 week each). How
> would the searching application handle/merge results from different
> indexes?
>
>
> Regards,
> CW
>
> Just a postscript here to thank mailing list folks who have been
> providing us with guidance on Lucene all this time :)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message