tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Schultz <ch...@christopherschultz.net>
Subject Re: Massive Startup Time after Server Reboot
Date Tue, 21 May 2019 14:55:38 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Jerry,

On 5/20/19 14:05, Jerry Malcolm wrote:
> Andre,
> 
> Your theory sounds like a real possibility.  It would explain why
> it only blows up the first time after a reboot.  I'll test the
> theory tonight during off-peak use hours for the web sites I host.

If André's theory is correct, you will probably find that 2-3 months
ago the version of your virus scanner was upgraded, and all of the
whitelisted applications (like Tomcat) were lost.

If that's the case, you can probably re-whitelist Tomcat and get your
performance back. At least until the next upgrade.

- -chris

> On 5/20/2019 3:43 AM, André Warnier (tomcat) wrote:
>> On 20.05.2019 00:09, Jerry Malcolm wrote:
>>> Just an FYI.... this server has been in production a little
>>> over a year.  This TC load problem only started 2-3 months ago.
>>> So 'something' changed to cause this problem to manifest
>>> itself. It's been too long to try to correlate a specific 
>>> server change to the start of this problem. But I just wanted
>>> to point out that this problem has NOT been around since day 1
>>> on this server.
>> 
>> Just in case, since this is a Windows server .. 2-3 months ago,
>> an update of a virus scanner ? (of the intrusive kind which
>> pre-checks each file that wants to be opened) Checking may be
>> easy or not, depending on your access : disable the virus scanner
>> just the time to start tomcat and check.
>> 
>>> 
>>> Jerry
>>> 
>>> On 5/19/2019 5:03 PM, Jerry Malcolm wrote:
>>>> Rainer,
>>>> 
>>>> No change with the urandom parm.  I am attaching a portion of
>>>> the Catalina log.  The first half shows between 8 and 15
>>>> seconds to deploy each app for a single virtual host (there
>>>> are no war files, the app is already exploded in the appbase 
>>>> dir).  I have quite a few virtual hosts with several apps
>>>> each.  That initial server start took 21+ minutes. I then
>>>> restarted the TC service and got the ~.5 sec start per webapp
>>>> shown in the 2nd half of the log below.
>>>> 
>>>> Can you refresh me on how to capture the stack dumps you
>>>> suggested? It's been a while....
>>>> 
>>>> Thx.
>>>> 
>>>> Jerry
>>>> 
>>>> First start of Tomcat after server reboot 
>>>> -------------------------------------------------------------------
- --
>>>>
>>>> 
[C:\domains\<myDomain>.com\webapps\JSPWiki.war] has finished in
>>>> [8,579] ms tory [C:\domains\<myDomain>.com\webapps\cis] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\cis] has finished
>>>> in [11,486] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\gallery] irectory
>>>> [C:\domains\<myDomain>.com\webapps\gallery] has finished in 
>>>> [9,204] ms tory [C:\domains\<myDomain>.com\webapps\gl] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\gl] has finished
>>>> in [8,469] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\idmanager] irectory
>>>> [C:\domains\<myDomain>.com\webapps\idmanager] has finished in
>>>> [8,689] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\itemtrack] irectory
>>>> [C:\domains\<myDomain>.com\webapps\itemtrack] has finished in
>>>> [6,907] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\malcolment] irectory
>>>> [C:\domains\<myDomain>.com\webapps\malcolment] has finished 
>>>> in [8,469] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\notebook] irectory
>>>> [C:\domains\<myDomain>.com\webapps\notebook] has finished in
>>>> [10,189] ms tory [C:\domains\<myDomain>.com\webapps\order] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\order] has
>>>> finished in [8,501] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\payment] irectory
>>>> [C:\domains\<myDomain>.com\webapps\payment] has finished in 
>>>> [14,209] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\projectmanager] irectory
>>>> [C:\domains\<myDomain>.com\webapps\projectmanager] has 
>>>> finished in [9,018] ms [C:\Tomcat
>>>> 9.0\conf\Catalina\myridetx.net\manager.xml] ptor [C:\Tomcat
>>>> 9.0\conf\Catalina\myridetx.net\manager.xml] has finished in
>>>> [62] ms
>>>> 
>>>> Restart of Tomcat 
>>>> -------------------------------------------------------------------
- --
>>>>
>>>> 
tory [C:\domains\<myDomain>.com\webapps\cis]
>>>> irectory [C:\domains\<myDomain>.com\webapps\cis] has finished
>>>> in [594] ms tory [C:\domains\<myDomain>.com\webapps\gallery] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\gallery] has
>>>> finished in [547] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\gl] irectory
>>>> [C:\domains\<myDomain>.com\webapps\gl] has finished in [562]
>>>> ms tory [C:\domains\<myDomain>.com\webapps\idmanager] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\idmanager] has
>>>> finished in [578] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\itemtrack] irectory
>>>> [C:\domains\<myDomain>.com\webapps\itemtrack] has finished in
>>>> [547] ms tory [C:\domains\<myDomain>.com\webapps\malcolment] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\malcolment] has
>>>> finished in [579] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\notebook] irectory
>>>> [C:\domains\<myDomain>.com\webapps\notebook] has finished in
>>>> [547] ms tory [C:\domains\<myDomain>.com\webapps\order] 
>>>> irectory [C:\domains\<myDomain>.com\webapps\order] has
>>>> finished in [531] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\payment] irectory
>>>> [C:\domains\<myDomain>.com\webapps\payment] has finished in 
>>>> [562] ms tory
>>>> [C:\domains\<myDomain>.com\webapps\projectmanager] irectory
>>>> [C:\domains\<myDomain>.com\webapps\projectmanager] has 
>>>> finished in [531] ms
>>>> 
>>>> On 5/18/2019 3:18 PM, Rainer Jung wrote:
>>>>> Most likely it hangs waiting for enough entropy for random
>>>>> number generator seeding.
>>>>> 
>>>>> Try whether the problem goes away if you add
>>>>> 
>>>>> -Djava.security.egd=file:/dev/urandom
>>>>> 
>>>>> to you process flags. If you are using older Java than Java
>>>>> 8 (not possible for Tomcat 9 but just in case you also have
>>>>> older software stacks running), then it would be
>>>>> 
>>>>> -Djava.security.egd=file:/dev/./urandom
>>>>> 
>>>>> The cryptic /./ is not a typo, you would need it for Java
>>>>> before Java 8.
>>>>> 
>>>>> If this doesn't help, then you should try capturing a few
>>>>> stack dumps (thread dumps) during the long startup time.
>>>>> People here can help interpret them.
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> Rainer
>>>>> 
>>>>> Am 18.05.2019 um 21:55 schrieb Jerry Malcolm:
>>>>>> This is a weird one.  It started a few months ago.   I
>>>>>> have TC 9 running on Windows Server 16.  After I reboot
>>>>>> the entire server, Tomcat takes forever on startup. It 
>>>>>> normally starts in about 30 seconds.  But after a server
>>>>>> reboot it takes up to 15 minutes... chugging along at a
>>>>>> snail's pace starting up all of the apps on all of the 
>>>>>> virtual hosts.  It always finally gets there with
>>>>>> everything successfully running. Other servers on the
>>>>>> same box (Apache, JAMES, ISC BIND, MySQL) don't have any
>>>>>> problem starting up. CPU, Disk, Memory, etc. usages are
>>>>>> barely showing on the performance graphs. There's nothing
>>>>>> in the Catalina log or system.err other than showing a
>>>>>> couple of minute gap in the time stamp between each app
>>>>>> as it starts up. If I need to reboot again later, it
>>>>>> boots again in about 30 sec as expected.
>>>>>> 
>>>>>> First question... any ideas off the top of your head that
>>>>>> might be causing this?
>>>>>> 
>>>>>> Second question... is there any other logging I can turn
>>>>>> on that might explain what TC is blocking on?
>>>>>> 
>>>>>> Thanks.
>>>>>> 
>>>>>> Jerry
>>>>> 
>>>>> ------------------------------------------------------------------
- ---
>>>>>
>>>>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>>> For additional commands, e-mail:
>>>>> users-help@tomcat.apache.org
>>>>> 
>>>> 
>>>> -------------------------------------------------------------------
- --
>>>>
>>>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail:
>>>> users-help@tomcat.apache.org
>>>> 
>>> 
>>> --------------------------------------------------------------------
- -
>>>
>>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>>
>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
> 
> ---------------------------------------------------------------------
>
> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlzkEWoACgkQHPApP6U8
pFinLRAAttjrmJnG6lwB3pd7kT5IpBk5H/MwJdXvEX8/S/k6MLLgWqWmqQhgRQAZ
UeVqJFJFDCrgXymxXNI2f7k3saVA+2Lxafjuft9WQGKs+mTSQR0s82VG6Iko5JzL
dZxA5XD+SSU6PafTMQ1hmlVMMDeug+ONqMcUY2XFoZSV6TUk6KqjMiod74We7/b8
7PSqI4VWSS6fA4/ZoPJNRvGLXtYDu1obVX3YysMb4tSky/5OenJX6rz0mwMenY0z
F2p2G2ObNlLKUq7DuAhBcLHB4lbkEv+20Zv6hqMGPlFB8rjiOpUpL86/gfdJY29z
KGgWLAw1M4rw2n0GjVihcfYfjVBA931wWwkaXpa6cXCrcyjOcgCH+OlhIqjtoHkW
w6mM9m99LqrWRwqqfiHDDxqywdqukfGANwi1QvaPgVkd0+FwGkinh5G3P9au3cLs
JKdpGgEog1hJnjkkXWwROlozKiKjMD82V474E4mwjw7nqPI/YXXL5E8VTlhm11JB
OWDZ8yNgwA//Aw7JbGdAUPqRSsSqrKIyGYr4voDd6nhs+83R/7z27IyIxgw/kzFU
6xRFOorztK5dOR5TqPZYB67qUL39++fpdayTYDs7f5Y8guajOsWgVTrvWo3bxCkg
CxUE7V91tWJYJKlxgbFZyZp4vvSzabf4dq5fxKhPTkYH9pmHhCs=
=z1un
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message