lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Solr OutOfMemory but no heap and dump and oo_solr.sh is not triggered
Date Wed, 03 Jun 2015 12:23:17 GMT
We will have to a find a way to deal with this long term. Browsing the code
I can see a variety of places where problem exception handling has been
introduced since this all was fixed.

- Mark

On Wed, Jun 3, 2015 at 8:19 AM Mark Miller <markrmiller@gmail.com> wrote:

> File a JIRA issue please. That OOM Exception is getting wrapped in a
> RuntimeException it looks. Bug.
>
> - Mark
>
>
> On Wed, Jun 3, 2015 at 2:20 AM Clemens Wyss DEV <clemensdev@mysign.ch>
> wrote:
>
>> Context: Lucene 5.1, Java 8 on debian. 24G of RAM whereof 16G available
>> for Solr.
>>
>> I am seeing the following OOMs:
>> ERROR - 2015-06-03 05:17:13.317; [   customer-1-de_CH_1]
>> org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
>> java.lang.OutOfMemoryError: Java heap space
>>         at
>> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:854)
>>         at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:463)
>>         at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
>>         at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>>         at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>>         at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>>         at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>>         at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>>         at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>>         at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>>         at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>>         at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>>         at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>>         at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>>         at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>>         at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>>         at org.eclipse.jetty.server.Server.handle(Server.java:368)
>>         at
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>>         at
>> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
>>         at
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
>>         at
>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
>>         at
>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>>         at
>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>>         at
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
>>         at
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>>         at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>>         at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>>         at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> WARN  - 2015-06-03 05:17:13.319; [   customer-1-de_CH_1]
>> org.eclipse.jetty.servlet.ServletHandler; Error for
>> /solr/customer-1-de_CH_1/suggest_phrase
>> java.lang.OutOfMemoryError: Java heap space
>>
>> The full commandline is
>> /usr/local/java/bin/java -server -Xss256k -Xms16G
>> -Xmx16G -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90
>> -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
>> -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark
>> -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly
>> -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000
>> -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc
>> -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps
>> -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution
>> -XX:+PrintGCApplicationStoppedTime -Xloggc:/opt/solr/logs/solr_gc.log
>> -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC
>> -Dsolr.solr.home=/opt/solr/data -Dsolr.install.dir=/usr/local/solr
>> -Dlog4j.configuration=file:/opt/solr/log4j.properties
>> -jar start.jar -XX:OnOutOfMemoryError=/usr/local/solr/bin/oom_solr.sh
>> 8983 /opt/solr/logs OPTIONS=default,rewrite
>>
>> So I'd expect /usr/local/solr/bin/oom_solr.sh tob e triggered. But this
>> does not seem to "happen". What am I missing? Is it o to pull a heapdump
>> from Solr before killing/rebooting in oom_solr.sh?
>>
>> Also I would like to know what query parameters were sent to
>> /solr/customer-1-de_CH_1/suggest_phrase (which may be the reason fort he
>> OOM ...
>>
>>
>> --
> - Mark
> about.me/markrmiller
>
-- 
- Mark
about.me/markrmiller

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message