lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Huge load and long response times during search
Date Sat, 21 Nov 2009 03:15:50 GMT
Tom,

It looks like the machine might simply be running too many things.  If the load is around
1 when Solr is not running, and this is a dual-core server, it shows its already relatively
busy (cca 50% idle).  Your caches are not small, so I am guessing you either have to have
a relatively big heap, or your heap is not large enough and it's the GC that's causing high
CPU load.  If you are seeing Solr causing lots of IO, that's a sign the box doesn't have enough
memory for all those servers running comfortably on it.

 Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----
> From: Tomasz Kępski <tomasz@kepski.pl>
> To: solr <solr-user@lucene.apache.org>
> Sent: Fri, November 20, 2009 3:15:08 PM
> Subject: Huge load and long response times during search
> 
> Hi,
> 
> I'm using SOLR(1.4) to search among about 3,500,000 documents. After the server 
> kernel was updated to 64bit system has started to suffer.
> Our server has 8G of RAM and double Intel Core 2 DUO.
> We used to have average loads around 2-2,5. It was not as good as it should but 
> as long HTTP response times was acceptable we do not care to much ;-)
> 
> Since few days avg loads are usually around 6, sometimes goes even to 20. PHP, 
> Mysql and Postgresql based application is rather fine, but when tries to access 
> SOLR it takes ages to load page. In top java process (Jetty) takes 200-250% of 
> CPU, iotop shows that most of the disk operations are done by SOLR threads as 
> well.
> 
> When we do shut down Jetty load goes down to 1,5 or even less than 1.
> 
> My index has ~12G below is a part of my solrconf.xml:
> 
> 
>   1024
>   
>      class="solr.LRUCache"
>      size="16384"
>      initialSize="4096"
>      autowarmCount="4096"/>
>   
>      class="solr.LRUCache"
>      size="16384"
>      initialSize="4096"
>      autowarmCount="1024"/>
>   
>      class="solr.LRUCache"
>      size="16384"
>      initialSize="16384"
>      autowarmCount="0"/>
>   true
>   true
>   40
>   200
>   
>   
>     
>       solr 0 
> name="rows">10 
>       solr price 
> name="start">0 10 
>       solr rekomendacja 
> name="start">0 10 
>       static newSearcher warming query from 
> solrconfig.xml
>     
>   
>   
>     
>       fast_warm 0 
> name="rows">10 
>       static firstSearcher warming query from 
> solrconfig.xml
>     
>   
>   false
> 
> 
> 
>   
>     dismax
>     explicit
>     0.01
>     
>        name^90.0 scategory^450.0 brand^90.0 text^0.01 description^30
>     
>     
>     
>     
>     
>     
>        brand,description,id,name,price,score
>     
>     
>        4<100% 5<90%
>     
>     100
>     *:*
>   
> 
> 
> sample query parameters from log looks like this:
> 
> 2009-11-20 21:07:15 org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/select 
> params={spellcheck=true&wt=json&rows=20&json.nl=map&start=520&facet=true&spellcheck.collate=true&fl=id,name,description,preparation,url,shop_id&q=camera&qt=dismax&version=1.3&hl.fl=name,description,atributes,brand,url&facet.field=shop_id&facet.field=brand&hl.fragsize=200&spellcheck.count=5&hl.snippets=3&hl=true}

> hits=3784 status=0 QTime=83
> 2009-11-20 21:07:15 org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/spellCheckCompRH 
> params={spellcheck=true&wt=json&rows=20&json.nl=map&start=520&facet=true&spellcheck.collate=true&fl=id,name,description,preparation,url,shop_id&q=camera&qt=dismax&version=1.3&hl.fl=name,description,atributes,brand,url&facet.field=shop_id&facet.field=brand&hl.fragsize=200&spellcheck.count=5&hl.snippets=3&hl=true}

> hits=3784 status=0 QTime=16
> 
> And at last the question ;-)
> How to speed up the search?
> Which parameters should I check first to find out what is the bottleneck?
> 
> Sorry for verbose entry but I would like to give as clear point of view as 
> possible
> 
> Thanks in advance,
> Tom


Mime
View raw message