httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 54502] New: Apache deadlock on epoll_ctl error (1000 process limit)
Date Tue, 29 Jan 2013 10:47:03 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=54502

            Bug ID: 54502
           Summary: Apache deadlock on epoll_ctl error (1000 process
                    limit)
           Product: Apache httpd-2
           Version: 2.2.15
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mpm_prefork
          Assignee: bugs@httpd.apache.org
          Reporter: etienne.champetier@free.fr
    Classification: Unclassified

Hi

With kernel 3.2.9 (included) to 3.2.17 (excluded) there was an arbitrary
limitation on epoll path (1000) which cause apache to deadlock when having
1001+ process. The first patch is
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=28d82dc1c4edbc352129f97f4ca22624d1fe61de,
which put the limit to 1000, and the second patch is
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=93dc6107a76daed81c07f50215fa6ae77691634f,
which doesn't limit epoll for non-nested path (so apache work again).


This limitation show a bug in apache which lead to a deadlock: if a httpd
process get an error when doing epoll_ctl, it continue to run, and if he get
the accept_mutex, epoll_wait will return 0 because epoll_ctl just failed, and
apache will be blocked.
Here follow a small strace of the 1001 process:
-epoll_create1(O_CLOEXEC)    = 39
-epoll_ctl(39, EPOLL_CTL_ADD, 6, {EPOLLIN, {u32=1010443880,
u64=140193037952616}}) = -1 EINVAL (Invalid argument)
-epoll_ctl(39, EPOLL_CTL_ADD, 4, {EPOLLIN, {u32=1010443880,
u64=140193037952616}}) = -1 EINVAL (Invalid argument)
-semop(14385470, {{0, -1, SEM_UNDO}}, 1 <unfinished ...>
<... semop resumed> )       = 0
-epoll_wait(39,  <unfinished ...>
<... epoll_wait resumed> {}, 2, 10000) = 0


To reproduce:
-get a kernel with the limitation (3.2.9 to 3.2.16 for the 3.2 branch)
-configure httpd to listen on at least 2 ports (80 and 81) so that it use
accept_mutex
-configure httpd to "StartServers 1001"
-start it with strace -f /etc/init.d/httpd start > ~/debug.log
-make a lot of request until it stop responding


The httpd process that fail to epoll_ctl should kill it self or retry
epoll_ctl.


This bug was uncovered on a centos 6.3 with httpd 2.2.15 and a 3.2.13 kernel,
but i've read other thread speaking of the 1000 httpd process limit on
ubuntu...
https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1028470 (so still
present in 2.2.22 for sure)

I've put normal severity because by updating the kernel apache work again.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message