lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: batch indexing takes more time than shown on SOLR output --> something to do with IO?
Date Mon, 14 Jan 2008 19:14:39 GMT
Re monitoring IO activity.... iostat, vmstat, sar and such under Linux, for example.

Yes, Solr doesn't count how long it takes to send the response back to the client, so if the
response is large and/or network is slow, the actual number is going to be higher than the
number that Solr logs.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

----- Original Message ----
From: Britske <gbrits@gmail.com>
To: solr-user@lucene.apache.org
Sent: Monday, January 14, 2008 10:56:24 AM
Subject: batch indexing takes more time than shown on SOLR output  --> something to do
with IO?


I have a batch program which inserts items in a solr/lucene index. 
all is going fine and I get update messages in the console like: 

14-jan-2008 16:40:52
 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {add=[10485, 10488, 10489, 10490, 10491, 10495, 10497, 10498,
 ...(42
more)
]} 0 875

However, when timing this instruction on the client-side (I use SOlrJ
 -->
req.process(server)) I get totally different numbers (in the beginning
 the
client-side measured time is about 2 seconds on average but after some
 time
this time goes up to about 30-40 seconds, altough the solr-outputted
 time
stays between 0.8-1.3 seconds? 

Does this have anything to do with costly IO-activity that is accounted
 for
in the SOLR output? If this is true, what tool do you recommend using
 to
monitor IO-activity?

Thanks, 
Geert-Jan 
-- 
View this message in context:
 http://www.nabble.com/batch-indexing-takes-more-time-than-shown-on-SOLR-output----%3E-something-to-do-with-IO--tp14804471p14804471.html
Sent from the Solr - User mailing list archive at Nabble.com.





Mime
View raw message