lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Rogers <chris.rog...@bodleian.ox.ac.uk>
Subject RE: SolrCloud: A previous ephemeral live node still exists
Date Wed, 31 Aug 2016 08:34:02 GMT
Just pinging this again as I sent it late last night. Would be great if someone could help
with this. It's got me totally stumped...


Chris Rogers
Digital Projects Manager
Bodleian Digital Library Systems and Services
chris.rogers@bodleian.ox.ac.uk

________________________________________
From: Chris Rogers [chris.rogers@bodleian.ox.ac.uk]
Sent: 30 August 2016 21:34
To: solr-user@lucene.apache.org
Subject: SolrCloud: A previous ephemeral live node still exists

Hi all,

I'm trying to create a SolrCloud setup with Vagrant boxes using Solr 6.2.0 and Zookeeper 3.4.8

I managed to get this to work perfectly with Solr 6.1.0, but I'm not able to start more than
one node in cloud mode with Solr 6.2.0.

I have four VMs connected on a private network of Vagrant boxes, one running Zookeeper, and
three Solr nodes.

The first Solr node connects to Zookeeper as expected. I can access the admin and see the
Cloud info there.

But when I try and connect with a second node, something goes wrong.

I get the following error in the Solr log:


2016-08-30 20:12:12.292 INFO  (main) [   ] o.e.j.u.log Logging initialized @490ms

2016-08-30 20:12:12.551 INFO  (main) [   ] o.e.j.s.Server jetty-9.3.8.v20160314

2016-08-30 20:12:12.574 INFO  (main) [   ] o.e.j.d.p.ScanningAppProvider Deployment monitor
[file:///home/vagrant/solr-6.2.0/server/contexts/] at interval 0

2016-08-30 20:12:12.906 INFO  (main) [   ] o.e.j.w.StandardDescriptorProcessor NO JSP Support
for /solr, did not find org.apache.jasper.servlet.JspServlet

2016-08-30 20:12:12.923 WARN  (main) [   ] o.e.j.s.SecurityHandler ServletContext@o.e.j.w.WebAppContext@5383967b{/solr,file:///home/vagrant/solr-6.2.0/server/solr-webapp/webapp/,STARTING}{/home/vagrant/solr-6.2.0/server/solr-webapp/webapp}
has uncovered http methods for path: /

2016-08-30 20:12:12.936 INFO  (main) [   ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
WebAppClassLoader=1465085305@57536d79

2016-08-30 20:12:12.962 INFO  (main) [   ] o.a.s.c.SolrResourceLoader JNDI not configured
for solr (NoInitialContextEx)

2016-08-30 20:12:12.962 INFO  (main) [   ] o.a.s.c.SolrResourceLoader using system property
solr.solr.home: /home/vagrant/solr-6.2.0/server/solr

2016-08-30 20:12:12.963 INFO  (main) [   ] o.a.s.c.SolrResourceLoader new SolrResourceLoader
for directory: '/home/vagrant/solr-6.2.0/server/solr'

2016-08-30 20:12:12.963 INFO  (main) [   ] o.a.s.c.SolrResourceLoader JNDI not configured
for solr (NoInitialContextEx)

2016-08-30 20:12:12.963 INFO  (main) [   ] o.a.s.c.SolrResourceLoader using system property
solr.solr.home: /home/vagrant/solr-6.2.0/server/solr

2016-08-30 20:12:12.987 INFO  (main) [   ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider

2016-08-30 20:12:13.017 INFO  (main) [   ] o.a.s.c.c.ConnectionManager Waiting for client
to connect to ZooKeeper

2016-08-30 20:12:13.118 INFO  (zkCallback-1-thread-1) [   ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@25b8acce name:ZooKeeperConnection Watcher:172.28.128.3:2181
got event WatchedEvent state:SyncConnected type:None path:null path:null type:None

2016-08-30 20:12:13.119 INFO  (main) [   ] o.a.s.c.c.ConnectionManager Client is connected
to ZooKeeper

2016-08-30 20:12:13.119 INFO  (main) [   ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider

2016-08-30 20:12:13.130 INFO  (main) [   ] o.a.s.s.SolrDispatchFilter Loading solr.xml from
SolrHome (not found in ZooKeeper)

2016-08-30 20:12:13.133 INFO  (main) [   ] o.a.s.c.SolrXmlConfig Loading container configuration
from /home/vagrant/solr-6.2.0/server/solr/solr.xml

2016-08-30 20:12:13.233 INFO  (main) [   ] o.a.s.c.CorePropertiesLocator Config-defined core
root directory: /home/vagrant/solr-6.2.0/server/solr

2016-08-30 20:12:13.264 INFO  (main) [   ] o.a.s.c.CoreContainer New CoreContainer 182259421

2016-08-30 20:12:13.265 INFO  (main) [   ] o.a.s.c.CoreContainer Loading cores into CoreContainer
[instanceDir=/home/vagrant/solr-6.2.0/server/solr]

2016-08-30 20:12:13.266 WARN  (main) [   ] o.a.s.c.CoreContainer Couldn't add files from /home/vagrant/solr-6.2.0/server/solr/lib
to classpath: /home/vagrant/solr-6.2.0/server/solr/lib

2016-08-30 20:12:13.281 INFO  (main) [   ] o.a.s.h.c.HttpShardHandlerFactory created with
socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize
: 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries
: false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,

2016-08-30 20:12:13.507 INFO  (main) [   ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler
HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true

2016-08-30 20:12:13.511 INFO  (main) [   ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory

2016-08-30 20:12:13.512 INFO  (main) [   ] o.a.s.l.LogWatcher Registering Log Listener [Log4j
(org.slf4j.impl.Log4jLoggerFactory)]

2016-08-30 20:12:13.519 INFO  (main) [   ] o.a.s.c.ZkContainer Zookeeper client=172.28.128.3:2181

2016-08-30 20:12:13.535 INFO  (main) [   ] o.a.s.c.ZkController Added new OnReconnect listener
org.apache.solr.cloud.ZkController$$Lambda$5/306612792@1aa7ecca

2016-08-30 20:12:13.538 INFO  (main) [   ] o.a.s.c.c.ConnectionManager Waiting for client
to connect to ZooKeeper

2016-08-30 20:12:13.549 INFO  (zkCallback-4-thread-1-processing-n:10.0.2.15:8983_solr) [ 
 ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6ece6797
name:ZooKeeperConnection Watcher:172.28.128.3:2181 got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None

2016-08-30 20:12:13.549 INFO  (main) [   ] o.a.s.c.c.ConnectionManager Client is connected
to ZooKeeper

2016-08-30 20:12:13.620 INFO  (main) [   ] o.a.s.c.c.ZkStateReader Updating cluster state
from ZooKeeper...

2016-08-30 20:12:13.627 INFO  (main) [   ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties

2016-08-30 20:12:13.634 INFO  (main) [   ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (1)

2016-08-30 20:12:43.674 ERROR (main) [   ] o.a.s.s.SolrDispatchFilter Could not start Solr.
Check solr/home property and the logs

2016-08-30 20:12:43.741 ERROR (main) [   ] o.a.s.c.SolrCore null:org.apache.solr.common.SolrException:
A previous ephemeral live node still exists. Solr cannot continue. Please ensure that no other
Solr process using the same port is running already.

at org.apache.solr.cloud.ZkController.checkForExistingEphemeralNode(ZkController.java:741)

at org.apache.solr.cloud.ZkController.init(ZkController.java:675)

at org.apache.solr.cloud.ZkController.<init>(ZkController.java:419)

at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)

at org.apache.solr.core.CoreContainer.load(CoreContainer.java:437)

at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:158)

at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:134)

at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:856)

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:348)

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1379)

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1341)

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:517)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:41)

at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188)

at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:499)

at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:147)

at org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:180)

at org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:458)

at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:64)

at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:610)

at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:529)

at org.eclipse.jetty.util.Scanner.scan(Scanner.java:392)

at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:313)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:150)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:561)

at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:236)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)

at org.eclipse.jetty.server.Server.start(Server.java:405)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)

at org.eclipse.jetty.server.Server.doStart(Server.java:372)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)

at java.security.AccessController.doPrivileged(Native Method)

at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.eclipse.jetty.start.Main.invokeMain(Main.java:214)

at org.eclipse.jetty.start.Main.start(Main.java:457)

at org.eclipse.jetty.start.Main.main(Main.java:75)


2016-08-30 20:12:43.744 INFO  (main) [   ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init()
done

2016-08-30 20:12:43.762 INFO  (main) [   ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@5383967b{/solr,file:///home/vagrant/solr-6.2.0/server/solr-webapp/webapp/,AVAILABLE}{/home/vagrant/solr-6.2.0/server/solr-webapp/webapp}

2016-08-30 20:12:43.771 INFO  (main) [   ] o.e.j.s.ServerConnector Started ServerConnector@305b7c14{HTTP/1.1,[http/1.1]}{0.0.0.0:8983}

2016-08-30 20:12:43.771 INFO  (main) [   ] o.e.j.s.Server Started @31972ms

It does start in the end, but it's not working properly, I just get errors in the Solr admin.

The only reference I can find to the the error:

"A previous ephemeral live node still exists. Solr cannot continue. Please ensure that no
other Solr process using the same port is running already."

,was this Jira ticket, but it suggests that the issue is resolved in 6.2.0

https://issues.apache.org/jira/browse/SOLR-8777

Has anyone else experienced this issue? Any hints on resolving?


Chris Rogers
Digital Projects Manager
Bodleian Digital Library Systems and Services
chris.rogers@bodleian.ox.ac.uk<mailto://chris.rogers@bodleian.ox.ac.uk>

Mime
View raw message