Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <49A45F53.9060508@iki.fi>
Date: Tue, 24 Feb 2009 22:57:55 +0200
From: Juha Laiho <Juha.Laiho@iki.fi>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.8.1.19) Gecko/20081209 Thunderbird/2.0.0.19 Mnenhy/0.7.6.666
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: very slow class loading on initial JSP/servlet request after
 restart
References: <499DEA05.4040501@ims.net>
 <499EB1D6.7020302@christopherschultz.net> <499ED728.4080909@ims.net>
In-Reply-To: <499ED728.4080909@ims.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

So, looks like I'm late to the party, but will crash in nonetheless.


Sam Hokin wrote:
> Christopher Schultz wrote:
>> On 2/19/2009 6:23 PM, Sam Hokin wrote:
>>> The problem, which spontaneously appeared a few days before Christmas on
>>> this one server, is that the initial request of a JSP or servlet takes
>>> AGES to respond, usually exceeding several minutes.
> 
> Well, I have used a number of diagnostics, like jconsole, jstat and
> jhat, but haven't been able to glean anything informative from them
> about my issue.  I've run the -Xloggc JVM option, and watched it garbage
> collect, again not informing me of anything.  Yes, if I could figure out
> what the JVM is actually DOING doing the long wait, I'd hopefully be
> able to deal with it, but I honestly haven't been able to figure out how
> to do that.  What do you suggest?
> 
>> I'll cast my hat into the ring: I suspect you're loading a resource that
>> wants to looking an external resource (like an XML file with a remote
>> DTD/Schema and validation is on in the parser) and the DNS is tripping
>> you up.
> 
> No, there are no external references in my Java package.  It's all just
> local class files.

Still it sounds like a network (DNS) issue, but it does not pay to just
argue, more information is needed.

One tool that I haven't yet seen suggested is 'strace', the Linux system
call tracer. This will show all the calls your application makes to the
operating system. As you say the application is mostly idle during the
delay, it is, in one way or another, waiting for some OS service to
complete. 'strace' should provide you with timestamped information on
what OS services were called, with which arguments, and how long did
it take for them to return with results. 'strace' will leave you with
a huge file (or a set of huge files, depending on the options you use),
and going through them will take some time - but you'll most likely
also find what causes the delay.

If you decide to go this way, trim down your application to the minimum
where you can still see the delay; with this you hopefully can get
at least some reduction to the amount of log data written. Even though it
is possible to temporarily attach strace to a running process (like
the Tomcat at the moment when it is hung), I'd recommend starting the
tomcat under truss (with -f or -ff to make it also trace new processes
created), because this way it is easier to trace back the file descriptor
numbers accessed.
-- 
..Juha

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org