lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Update of "SolrPerformanceFactors" by jayqhacker
Date Mon, 26 Nov 2012 18:51:35 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceFactors" page has been changed by jayqhacker:
http://wiki.apache.org/solr/SolrPerformanceFactors?action=diff&rev1=31&rev2=32

Comment:
Add note about reducing memory usage with commits during indexing.

  For bulk updating from a Java client,
  consider using the [[http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html|StreamingUpdateSolrServer.java|StreamingUpdateSolrServer]]
which streams updates over multiple connections using multiple threads.
  
+ Reducing the frequency of automatic commits or disabling them entirely may speed indexing.
 Beware that this can lead to increased memory usage, which can cause performance issues of
its own, such as excessive swapping or garbage collection.
+ 
  == RAM Usage Considerations ==
  
  === OutOfMemoryErrors ===
@@ -155, +157 @@

  
  Note that several different "add" commands can be running simultaneously (in different threads).
The more threads, the greater the memory usage.
  
+ When indexing, memory usage will grow with the number of documents indexed until a commit
is performed.  A commit (including a soft commit) will free up almost all heap memory.  To
avoid very large heaps and associated garbage collection pauses during indexing, perform a
manual (soft) commit periodically, or consider enabling `autoCommit` (or `autoSoftCommit`)
in [[SolrConfigXml#Update_Handler_Section|solrconfig.xml]].
+ 

Mime
View raw message