lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <>
Subject [Solr Wiki] Update of "SolrPerformanceFactors" by Mark Miller
Date Sat, 05 Nov 2011 12:50:09 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceFactors" page has been changed by Mark Miller:

  == Optimization Considerations ==
- You may want to optimize an index whenever practical -- ie: if you build your index once,
and then never modify it.
+ You may want to optimize an index in certain situations -- ie: if you build your index once,
and then never modify it.
- If your index is receiving a steady stream of modifications, then consider the following
+ If you have a rapidly changing index, rather than optimizing, you likely simply want to
use a lower merge factor. Optimizing is very expensive, and if the index is constantly changing,
the slight performance boost will not last long. The tradeoff is not often worth it for a
non static index.
+ In a master slave setup, sometimes you may also want to optimize on the master so that slaves
serve from a single segment index. This will can greatly increase the time to replicate the
index though, so this is often not desirable either.
-    * As more segments are added to the index, query performace will degrade slightly.  Automatic
segment merging by Lucene will set an upper bound on the number of segments created though.
-    * Auto-warming time will grow since it's normally dependent on doing searches. 
-    * The first distribution after an optimization will take longer than subsequent ones.
See [[CollectionDistribution|Collection Distribution]] for more information.
-    * During optimization the file size of the index doubles, but returns to it's original
size or even slightly less.
-    * If you can, make sure that you do not have multiple concurrent producers of documents
calling commit(). Multiple concurrent commits will cause a large performance degradation.

- Since optimizing an index saves all the segments in an index (about 7 files per segment)
into a single segment, optimizing an index helps avoid the "too many open files" problem,
i.e. running out of file descriptors, which is mentioned in an [[|ONJava
  == Updates and Commit Frequency Tradeoffs ==

View raw message