lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branham, Jeremy [HR]" <Jeremy.D.Bran...@sprint.com>
Subject FW: suspect SOLR query from D029 (SOLR master)
Date Fri, 30 May 2014 16:51:30 GMT
We saw the file descriptors peak out and  full GCs running causing DOS on our SOLR servers
this morning.
*         Does this stack trace give enough information for some ideas?

*         solr-spec
4.5.1-SNAPSHOT
*         solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
*         lucene-spec
4.5.1
*         lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20,334 ERROR - SolrDispatchFilter         - null:org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,334 ERROR - SolrCore                   - org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http://myserver.com:8080/svc/solr/wdsp
2014-05-30 10:24:20,333 ERROR - SolrCore                   - org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrCore                   - org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,336 ERROR - SolrDispatchFilter         - null:org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrDispatchFilter         - null:org.apache.solr.common.SolrException:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server
at: http:// myserver.com:8080/svc/solr/wdsp



Jeremy D. Branham
Tel: **DOTNET

From: Worley, Chris [HR]
Sent: Friday, May 30, 2014 11:38 AM
To: Stoner, Susan [HR]; Branham, Jeremy [HR]
Cc: Barrett, Kevin B [HR]; Aldrich, Daniel J [HR]; Duncan, Horace W [HR]
Subject: suspect SOLR query from D029 (SOLR master)

Looks like a suspect SOLR query was executed at approx. 8:54am this morning on the SOLR master
server (D029) that caused the java GC to go through the roof.  See below:

2014-05-30 08:54:17,092 ERROR - SolrDispatchFilter         - null:java.lang.OutOfMemoryError:
GC overhead limit exceeded
        at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2377)
        at org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1698)
        at org.apache.lucene.index.TermContext.build(TermContext.java:95)
        at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:166)
        at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183)
        at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
        at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183)
        at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
        at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183)
        at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
        at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:183)
        at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
        at org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:82)
        at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:690)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
        at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1501)
        at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1367)
        at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
        at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
        at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)

Unfortunately, I don't have the SOLR query...

-CW

________________________________

This e-mail may contain Sprint proprietary information intended for the sole use of the recipient(s).
Any use by others is prohibited. If you are not the intended recipient, please contact the
sender and delete all copies of the message.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message