tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Warnier (tomcat) ...@ice-sa.com>
Subject Re: A complex issue concerning the application lifecycle, MBeans and Spring
Date Wed, 13 Jul 2016 13:41:02 GMT
On 13.07.2016 15:20, Jäkel, Guido wrote:
> Dear Developers,
>
> I have an elaborate issue migrating from Tomcat 6.0.41 to 8.0.36 (currently using java
8.0.66 on Gentoo Linux, but this is not of importance I think): It is related to the build-in
manager application used as a jmxproxy. And to initialization of the Spring framework during
an application lifecycle. But in core, I think it's not related to the manager app and it's
proxy feature itself. But to some MBean or lifecycle implementation details.
>
> First of all: In principle, all works well. If i start the tomcat from the scratch, all
applications will proper initialize and work as expected. If I restart any application by
different triggers (manager, touching the WAR or a sensitive file inside the docroot and so
on), it will be undeployed and deployied again - all as usual.
>
> But if there is a certain request to some MBean while the App is starting, there will
be a request to the application (via catalina-exec). And then, as a race condition, the initialization
of the Spring framework, esp. the bean wiring will fail with lots of exceptions because this
is not thread-safe.
>
> If this additional (failing) request is processed fast enough, the application will be
usable due to initialization via the normal servlet lifecycle request. But if it's as a race
condition in the other way round, the application will be not usable afterwards.
>
> The point now is: This requests to the MBeans are done requently (ever 10s) by our monitoring
system. And about 90% of the automatic deployments done by the CI system now will fail by
this race condition in spite of the deployed apps are well-done. Without any modification,
the same application is able to start by chance of no concurrent request to the MBean or (of
course) by disabling the Monitoring or factical preventing it's request also by a restart
of the whole tomcat, because the connectors are disabled until all apps are up as you'll know.
>
> This never happens with tomcat 6 used for many years. Therefore, there must be some regression
with an MBean that will allow to route a request to a not-fully-started application. I happens
with all our well-established applications which use the Spring framework -- regardess if
deployed via the CI because of normal, slightly modification or even with unchanged versions
running in production on Tomcat6. I can't give information about the behavior on Tomcat7.
>
> I think I was able to narrow it down to a specific MBean readout: If I send the request
".../manager/jmxproxy?qry=Catalina:j2eeType=Servlet,WebModule=*,*" before the Deployer logs
"[StandardContext] Reloading Context with name [foo] is completed", then it will trigger an
unwanted, additional thread.
>
> It sounds like it points into the right direction because the MBean related to the Servlet
is offering e.g. information about the containers lifecycle state.
>
> May you fix it or did you see any workaround? If you need any more details, please ask
about. Thank you in advance for your support.
>
>

Hi.
The explanation above, as well as the underlying issue, are way over my level of knowledge

of Tomcat and/or Java.

But, as a possible (and possibly temporary) workaround, and focusing on this bit :

"Without any modification, the same application is able to start by chance of no 
concurrent request to the MBean or (of course) by disabling the Monitoring or factical 
preventing it's request also by a restart of the whole tomcat, because the connectors are

disabled until all apps are up as you'll know."

So it would appear that if the Connectors are disabled, the monitoring system is not able

to reach Tomcat, and the problem does not occur then.

So would it not be possible to create some little piece of software, which would 
temporarily "suspend" or "disable" the Connectors, while the application update is taking

place, and then re-enable them when it's done ?

Of course, if the application update is also going though the Connectors, this may be a 
problem..




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message