httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Ziesmer <onl...@mark.ziesemer.com>
Subject [users@httpd] Re: AH00273: apr_proc_mutex_lock failed, possibly caused by cron, systemd or su
Date Wed, 30 Dec 2015 02:03:34 GMT
> Rainer Canavan <rainer.canavan <at> sevenval.com> writes:
> 
> > we've got an obscure problem with the apache httpd that was shipped
> > with CentOS 7.2. We perform automatic builds and updates via cron,
> > and, since the update to CentOS 7.2. The update script is triggered by
> > cron and stops, yum updates and starts the httpd. When the next cron
> > job that is run as the same user as the httpd (not the update job)
> > terminates, the httpd frequently fails, starting with a AH00273:
> > apr_proc_mutex_lock failed message, and then a never ending loop of
> > AH00272 messages, one from each httpd process that is forked, until
> > the listener process is stopped.
> 
> > Does anyone have any ideas what may cause these mutex errors?
> 
> Sorry - no ideas, but I'm having the exact same issue, and have been trying
> for the better part of a week to figure out what is causing this.

Discussed with thumbs on IRC.  Setting "Mutex sem" in my httpd.conf appears
to be a work-around for this.  Otherwise, it was unset, and would apparently
default to "Mutex default".

He suspected that "the apr package was botched by the RHEL maintainers". 
However, when I tried my compile-from-source work-around, it required a
source include of both apr and apr-util, so I would think that could be
discounted?  Or possibly lead to a worse issue in something lower-level
within RHEL/CentOS?  But still worked-around by using sem - and would
further explain why something like cron seems to need to be involved to
reproduce?

Frustratingly, reproducing this appears to be hit-and-miss.  Following-up on
my previous reply, I further did install mod_ssl and cloned over my complete
configurations from the problem instance to my test instance, and was still
unable to reproduce on a 2nd box with what I believe to be an identical
build and configuration.

We were curious if there was a way to determine if there was a way to know
which implementation was being selected by default (and problematic), as
both SYSVSEM and PTHREAD were defined in httpd -V (as shown below)?  I also
checked my compile-from-source work-around attempt, and using the
Apache-provided defaults, everything related to the Mutex appeared to be
identical.  (Same 2 defines shown in httpd -V, and not explicit Mutex
declaration in my httpd configuration files.)

Is there anything further I can provide to help with this, even though it
appears I may have a successful work-around?  (Should a bug be opened for
this on a tracker somewhere?)

$ httpd -V
Server version: Apache/2.4.6 (CentOS)
Server built:   Nov 19 2015 21:43:13
Server's Module Magic Number: 20120211:24
Server loaded:  APR 1.4.8, APR-UTIL 1.5.2
Compiled using: APR 1.4.8, APR-UTIL 1.5.2
Architecture:   64-bit
Server MPM:     worker
	threaded:     yes (fixed thread count)
		forked:     yes (variable process count)
Server compiled with....
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=256
 -D HTTPD_ROOT="/etc/httpd"
 -D SUEXEC_BIN="/usr/sbin/suexec"
 -D DEFAULT_PIDLOG="/run/httpd/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message