lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Update of "SolrPerformanceFactors" by YonikSeeley
Date Fri, 01 Apr 2011 18:51:10 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceFactors" page has been changed by YonikSeeley.
The comment on this change is: remove outdated benchmark.
http://wiki.apache.org/solr/SolrPerformanceFactors?action=diff&rev1=25&rev2=26

--------------------------------------------------

  
  Consult the documentation for the application server you are using (ie: !TomCat, Resin,
Jetty, etc...) for more information on how to configure page compression.
  
- == Embedded vs HTTP Post ==
+ == Indexing Performance ==
+ In general, adding many documents per update request is faster than one per update request.
  
+ For bulk updating from a Java client,
+ consider using the [[http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html|StreamingUpdateSolrServer.java|StreamingUpdateSolrServer]]
which streams updates over multiple connections using multiple threads.
- Using an [EmbeddedSolr] for indexing can be over 50% faster than one using XML messages
that are posted.  
- 
- For example it took 2:10:23 to index 3 million records and optimize, while it took 3:21:36
on the same machine to index using HTTP Post with 10 records/post or 2:37:17 with 200 records/post.
 If you consider that optimize is only one call, then the difference is slightly bigger. 
The machine for these sample numbers was a 3Ghz Pentium 4 desktop machine.
- 
- However the tradeoff is larger records/post requires greater memory footprint.  As the records/post
becomes higher it makes more sense to have separate threads for getting records from database/files
and another for posting the XML messages to Solr (could also double buffer).  
- 
- See [[http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/package-summary.html|java.util.concurrency
javadoc]] for more information on threading.
- 
- Also consider using the [[http://svn.apache.org/repos/asf/lucene/dev/trunk/solr/src/solrj/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.java|StreamingUpdateSolrServer.java]]
for bulk update request.  
- 
  
  == RAM Usage Considerations ==
  

Mime
View raw message