geronimo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevan Miller <kevan.mil...@gmail.com>
Subject Re: Startup time delay in Ubuntu server
Date Mon, 12 Jan 2009 23:38:48 GMT

On Jan 12, 2009, at 1:19 PM, eros wrote:

>
> Hi everyone,
>
> I'm having the same problem using Ubuntu 8.04 server and Geronimo  
> Tomcat6
> 2.1.3, with Java version 1.6.0_07.
>
> I can reproduce the problem quite easily (in fact, I can't get rid of
> it...): I installed geronimo on 3 different computers:
>
> - production server running Ubuntu 8.04 64bit (kernel 2.6.24-23)
> - development server running Ubuntu 8.04 32bit (kernel 2.6.24-23)
> - virtualbox (2.1.0) machine running Ubuntu 8.04 JeOS 32 bit (kernel
> 2.6.24-23)
>
> The problem manifests itself on all these computers: whenever I try  
> to start
> geronimo it gets stuck loading module 25/68, or at least that's what
> /usr/share/geronimo-tomcat6/var/log/geronimo.out says:
>
> Module 25/68 org.apache.geronimo.configs/tomcat6/2.1.3/car
>
> the last line of the other log file
> (/usr/share/geronimo-tomcat6/var/log/geronimo.log) while waiting for  
> Module
> 25/68 to load is:
>
> [SystemProperties] Setting
> Property=org.apache.catalina.STRICT_SERVLET_COMPLIANCE to Value=true
>
> Any suggestiong as to what I should try to fix this would be very much
> appreciated, and if yoy need more info just let me know.

I think you have a problem with system entropy that's causing a  
problem generating pseudo-random numbers.

I dusted off an Ubuntu 8.0.4 server which I haven't been using lately  
and was able to recreate your problem (or at least something that very  
closely approximates your problem --  a 90 second delay while starting  
the tomcat6 module).

While the server startup was stalled, I sent a kill -3 signal to the  
server process (think I mentioned doing this earlier in this thread)  
and got the following thread stack:

"main" prio=1 tid=0x0000000040117140 nid=0x14b0 runnable  
[0x00007fff12f12000..0x00007fff12f150e0]
         at java.io.FileInputStream.readBytes(Native Method)
         at java.io.FileInputStream.read(FileInputStream.java:194)
         at sun.security.provider.NativePRNG 
$RandomIO.readFully(NativePRNG.java:185)
         at sun.security.provider.NativePRNG 
$RandomIO.implGenerateSeed(NativePRNG.java:202)
         - locked <0x00007f51c4f48810> (a java.lang.Object)
         at sun.security.provider.NativePRNG$RandomIO.access 
$300(NativePRNG.java:108)
         at  
sun.security.provider.NativePRNG.engineGenerateSeed(NativePRNG.java:102)
         at java.security.SecureRandom.generateSeed(SecureRandom.java: 
475)
         at  
com 
.sun.net.ssl.internal.ssl.PKCS12KeyStore.getSalt(PKCS12KeyStore.java: 
475)
         at  
com 
.sun 
.net.ssl.internal.ssl.PKCS12KeyStore.calculateMac(PKCS12KeyStore.java: 
832)
         at  
com 
.sun 
.net.ssl.internal.ssl.PKCS12KeyStore.engineStore(PKCS12KeyStore.java: 
786)
         - locked <0x00007f51f6385438> (a  
com.sun.net.ssl.internal.ssl.PKCS12KeyStore)
         at java.security.KeyStore.store(KeyStore.java:1085)
         at  
org.apache.geronimo.crypto.KeystoreUtil.<clinit>(KeystoreUtil.java:101)
         at  
org 
.apache 
.geronimo 
.tomcat 
.TomcatManagerImpl.addSslConnectorAttributes(TomcatManagerImpl.java:450)
         at  
org 
.apache 
.geronimo.tomcat.TomcatManagerImpl.<clinit>(TomcatManagerImpl.java:101)
         at  
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
         at  
sun 
.reflect 
.NativeConstructorAccessorImpl 
.newInstance(NativeConstructorAccessorImpl.java:39)
         at  
sun 
.reflect 
.DelegatingConstructorAccessorImpl 
.newInstance(DelegatingConstructorAccessorImpl.java:27)
         at java.lang.reflect.Constructor.newInstance(Constructor.java: 
494)
         at  
org 
.apache 
.geronimo 
.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:948)
         at  
org 
.apache 
.geronimo 
.gbean 
.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java: 
268)
         at  
org 
.apache 
.geronimo 
.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:102)
         at  
org 
.apache 
.geronimo 
.gbean 
.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:124)
         at  
org 
.apache 
.geronimo 
.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:555)
         at  
org 
.apache 
.geronimo 
.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:379)
         at  
org 
.apache 
.geronimo 
.kernel 
.config 
.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:456)
         at  
org 
.apache 
.geronimo 
.kernel 
.config 
.KernelConfigurationManager.start(KernelConfigurationManager.java:188)
         at  
org 
.apache 
.geronimo 
.kernel 
.config 
.SimpleConfigurationManager 
.startConfiguration(SimpleConfigurationManager.java:562)
         - locked <0x00007f51c5183ee8> (a  
org.apache.geronimo.kernel.config.EditableKernelConfigurationManager)
         at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
         at  
sun 
.reflect 
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 
25)
         at java.lang.reflect.Method.invoke(Method.java:585)
         at  
org 
.apache 
.geronimo 
.gbean 
.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
         at  
org 
.apache 
.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:124)
         at  
org 
.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java: 
832)
         at  
org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57)
         at  
org 
.apache 
.geronimo 
.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35)
         at  
org 
.apache 
.geronimo 
.kernel 
.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96)
         at  
org.apache.geronimo.kernel.config.EditableConfigurationManager$ 
$EnhancerByCGLIB$$a351628e.startConfiguration(<generated>)
         at  
org 
.apache 
.geronimo.system.main.EmbeddedDaemon.doStartup(EmbeddedDaemon.java:162)
         at  
org 
.apache 
.geronimo.system.main.EmbeddedDaemon.execute(EmbeddedDaemon.java:79)
         at  
org 
.apache 
.geronimo 
.kernel 
.util 
.MainConfigurationBootstrapper.main(MainConfigurationBootstrapper.java: 
45)
         at  
org.apache.geronimo.cli.AbstractCLI.executeMain(AbstractCLI.java:67)
         at  
org.apache.geronimo.cli.daemon.DaemonCLI.main(DaemonCLI.java:30)

Once I entered the kill -3 (and the stacktraces printed), my server  
startup resumed (because I'd just generated some new entropy). The  
above is attempting to generate a pseudo-random number by reading  
from /dev/random.

I suggest that you verify (using kill -3) what your server is doing  
during the delay. If you see the above (or something pretty similar),  
then you're hitting the same problem.  Please let us know what you find.

One fix is to generate more entropy on your system (or live with this  
slow startup behavior). I've run into this issue previously on linux  
test systems... IIRC, I've seen people work around this with 'ln -s / 
dev/urandom /dev/random'. I'd guess that this isn't a very secure  
option. It's possible that we (Geronimo) might be able to do something  
about this behavior... I'd have to spend some quality time with  
org.apache.geronimo.crypto.KeyStoreUtil (see the static  
initializer...). If you or somebody else is able to do a bit of  
digging, that'd be great... Probably worth creating a Jira to track  
this...

--kevan 

Mime
View raw message