Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <498C71A4.9000404@kippdata.de>
Date: Fri, 06 Feb 2009 18:21:40 +0100
From: Rainer Jung <rainer.jung@kippdata.de>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de;
 rv:1.9.1b3pre) Gecko/20081204 Thunderbird/3.0b1
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: Deadlock situation detected/avoided with jk_log_lock
References: <21876381.post@talk.nabble.com>
In-Reply-To: <21876381.post@talk.nabble.com>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit

On 06.02.2009 18:13, fredk2 wrote:
> I was doing some stress test (with apache ab, 100 users, 100K requests) to
> compare an Apache prefork and worker mpm.  The test url is a simple hello
> servlet on Tomcat 6.0.x via mod_jk. On my Sparc Solaris 10 server with only
> the Apache set to worker mpm I see following error messages in my jk log:
>
> Apache/2.2.11 (Unix) with mod_jk/1.2.26 on Solaris 10.
> . . .
> [Thu Jan 08 11:42:28 2009] [error] (45)Deadlock situation detected/avoided:
> apr_global_mutex_lock(jk_log_lock) failed
> . . .
> [Thu Jan 08 11:42:29 2009] [emerg] (45)Deadlock situation detected/avoided:
> apr_proc_mutex_lock failed. Attempting to shutdown process gracefully.
> [Thu Jan 08 11:42:29 2009] [error] (45)Deadlock situation detected/avoided:
> apr_global_mutex_lock(jk_log_lock) failed
> . . .
>
> these errors do not appear to impact the test results and the jk log file
> seems complete.
>
> I can suppress the errors by choosing another Mutex in the Apache directive
> AcceptMutex, such as sysvsem or pthread.  For Solaris 10 the default mutex
> for worker MPM is fcntl.  Setting the Mutex sysvsem (also the default on
> Linux) marginally improves the request time.
>
> Can someone explain what exactly these errors means? when does it occur?
> I would have almost expect a "detected/avoided" to be a [warn] instead of an
> [error].
>
> I have seen the trail http://markmail.org/message/dedqpmrrkpa224ns but I'd
> like to hear updated experiences that people have with sysvsem mutexes on
> Solaris 10 - what is the better mutex?  sysvsme, posixsem, pthread **?
>
> any comment will be appreciated.

I experienced this too a couple of times and once wrote a small C 
program to reproduce the problem. On Solaris the algorithm to detect a 
possible deadlock is very careful and returns EDEADLOCK even in 
situations were you can mathematically prove, that a deadlock is not 
possible. This happens in a multi-threaded environment when more than 
one mutex is used.

Apache httpd and mod_jk use such a mutex and SSL also (so you can 
observe the same warnings without mod_jk only using SSL with httpd and 
doing stress tests).

In older JK versions this could lead to a hang, but we worked around 
that a couple of versions ago. I generally recommend the pthread mutex 
for Solaris which doesn't have the problem and seems to be robust 
despite warnings about pthread mutexes in very old versions of Solaris.

We even once had a discussion about changing the default httpd mutex on 
Solaris once, but I think that discussion didn't come to an end.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org