lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <>
Subject [Solr Wiki] Update of "SolrPerformanceFactors" by YonikSeeley
Date Fri, 01 Apr 2011 18:51:10 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrPerformanceFactors" page has been changed by YonikSeeley.
The comment on this change is: remove outdated benchmark.


  Consult the documentation for the application server you are using (ie: !TomCat, Resin,
Jetty, etc...) for more information on how to configure page compression.
- == Embedded vs HTTP Post ==
+ == Indexing Performance ==
+ In general, adding many documents per update request is faster than one per update request.
+ For bulk updating from a Java client,
+ consider using the [[||StreamingUpdateSolrServer]]
which streams updates over multiple connections using multiple threads.
- Using an [EmbeddedSolr] for indexing can be over 50% faster than one using XML messages
that are posted.  
- For example it took 2:10:23 to index 3 million records and optimize, while it took 3:21:36
on the same machine to index using HTTP Post with 10 records/post or 2:37:17 with 200 records/post.
 If you consider that optimize is only one call, then the difference is slightly bigger. 
The machine for these sample numbers was a 3Ghz Pentium 4 desktop machine.
- However the tradeoff is larger records/post requires greater memory footprint.  As the records/post
becomes higher it makes more sense to have separate threads for getting records from database/files
and another for posting the XML messages to Solr (could also double buffer).  
- See [[|java.util.concurrency
javadoc]] for more information on threading.
- Also consider using the [[|]]
for bulk update request.  
  == RAM Usage Considerations ==

View raw message