lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doss <itsmed...@gmail.com>
Subject Re: SOLR not starting after restart 2 node cloud setup
Date Thu, 20 Nov 2014 11:47:00 GMT
Dear Erick,

Forgive my ignorance.

Please find some of the details you required.

*have you looked at the solr logs?*

 > Sorry I haven't defined the log4j.properties file, so I don't have solr
logs. Since it requires tomcat restart I am planning to do it in next
restart.

But found the following in tomcat log

18-Nov-2014 11:27:29.028 WARNING [localhost-startStop-2]
org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads The web
application [/mima] appears to have started a thread named
[localhost-startStop-1-SendThread(10.236.149.28:2181)] but has failed to
stop it. This is very likely to create a memory leak. Stack trace of thread:
 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
 sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
 sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
 sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
 sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
 org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
 org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)


*How big are the cores?*

> We have 16 cores, out of it only 5 are big ones. Total size of all 16
cores is 10+ GB

*How many docs in the cores when the problem happens?*

1 core with 163 fields and 33,00,000 documents (Index size 2+ GB)
 4 cores with 3 fields and has 150,00,000 (approx) documents (1.2 to 1.5 GB)
remaining cores are 1,00,000 to 40,00,000 documents

*How much memory are you allocating the JVM? *

5GB for JVM, Total RAM available in the systems is 30 GB

*can you restart Tomcat without a problem?*

This problem is occurring in production, I never tried.


Thanks,
Doss.


On Wed, Nov 19, 2014 at 7:55 PM, Erick Erickson <erickerickson@gmail.com>
wrote:

> You've really got to provide details for us to say much
> of anything. There are about a zillion things that it could be.
>
> In particular, have you looked at the solr logs? Are there
> any interesting things in them? How big are the cores?
> How much memory are you allocating the JVM? How
> many docs in the cores when the problem happens?
> Before the nodes stop responding, can you restart
> Tomcat without a problem?
>
> You might review:
> http://wiki.apache.org/solr/UsingMailingLists
>
> Best,
> Erick
>
>
> On Wed, Nov 19, 2014 at 1:04 AM, Doss <itsmedoss@gmail.com> wrote:
> > I have two node SOLR (4.9.0) cloud with Tomcat (8), Zookeeper. At times
> > SOLR in Node 1 stops responding, to fix the issue I am restarting tomcat
> in
> > Node 1, but SOLR not starting up, but if I remove the solr cores in both
> > nodes and try restarting it starts working, and then I have to reindex
> the
> > whole data again. We are using this setup in production because of this
> > issue we are having 1 to 1.30 hours of service down time. Any suggestions
> > would be greatly appreciated.
> >
> > Thanks,
> > Doss.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message