httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [Bug 55117] New: GracefullTimeout for worker threads
Date Tue, 18 Jun 2013 21:35:55 GMT

            Bug ID: 55117
           Summary: GracefullTimeout for worker threads
           Product: Apache httpd-2
           Version: 2.2.22
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mpm_worker

We are observing a few deadlocking issues around process recycle time (one
example is below, but we are seeing others).  As an operational mitigation we
would like to be able to use either the GracefullTimeout parameter or a new
parameter to indicate how long we are willing to wait for the threads to
shutdown aka a deadlock timeout, before going into an ungraceful termination of
threads.  requests_this_child was at -1 in each case (and we have
MaxRequetsPerChild set)

I have read the source for 2.2.22 and 2.4.4 for the mpm_worker and the issue
will exist in all versions (as the listener thread will instruct threads to
terminate, perform its own shutdown then exit, the main thread will eventually
hit pthread_join and block. We are also looking at changing the openssl version
as we believe this is the source of the deadlocking.  However a operational
mitigation against any deadlock is still desirable. 

On platforms that support it pthread_tryjoin_np could perhaps be used ... 
timer_create may be more portable. This may mean the listener needs to be the
last to die or pass the pod fds other to the mainthread so it can perform a
ungraceful shutdown.

(gdb) info threads
  Id   Target Id         Frame 
  2    Thread 0x7f4c57fff700 (LWP 6028) "apache2" 0x00007f4c7c282be4 in
__lll_lock_wait () from /lib/x86_64-linux-gnu/
* 1    Thread 0x7f4c7cd3b760 (LWP 6008) "apache2" 0x00007f4c7c27d1f8 in
pthread_join () from /lib/x86_64-linux-gnu/

(gdb) thread 2
[Switching to thread 2 (Thread 0x7f4c57fff700 (LWP 6028))]
#0  0x00007f4c7c282be4 in __lll_lock_wait () from
(gdb) bt
#0  0x00007f4c7c282be4 in __lll_lock_wait () from
#1  0x00007f4c7c27e1e5 in _L_lock_883 () from
#2  0x00007f4c7c27e03a in pthread_mutex_lock () from
#3  0x00007f4c7c4ad890 in apr_thread_mutex_lock () from /usr/lib/
#4  0x00007f4c78469443 in ?? () from /lib/x86_64-linux-gnu/
#5  0x00007f4c78469d9c in ?? () from /lib/x86_64-linux-gnu/
#6  0x00007f4c7846ac81 in ERR_get_state () from
#7  0x00007f4c7846c369 in ERR_set_mark () from
#8  0x00007f4c784597fa in ?? () from /lib/x86_64-linux-gnu/
#9  0x00007f4c7846e598 in EVP_CipherInit_ex () from
#10 0x00007f4c78772f46 in ?? () from /lib/x86_64-linux-gnu/
#11 0x00007f4c7876ab7e in ?? () from /lib/x86_64-linux-gnu/
#12 0x00007f4c7876b819 in ?? () from /lib/x86_64-linux-gnu/
#13 0x00007f4c7876cb90 in ?? () from /lib/x86_64-linux-gnu/
#14 0x00007f4c78760054 in ?? () from /lib/x86_64-linux-gnu/
#15 0x00007f4c78761990 in ?? () from /lib/x86_64-linux-gnu/
#16 0x00007f4c7876ec13 in ?? () from /lib/x86_64-linux-gnu/
#17 0x00007f4c789b01a0 in ?? () from /usr/lib/apache2/modules/
#18 0x00007f4c789b0740 in ?? () from /usr/lib/apache2/modules/
#19 0x00007f4c7cd7ec86 in ap_rgetline_core (s=0x7f4c6c67a0d0, n=8192,
read=0x7f4c57ffed40, r=0x7f4c6c67a0a0, fold=0, bb=0x7f4c6c67b7e8)
    at /build/buildd/apache2-2.2.20/server/protocol.c:231
#20 0x00007f4c7cd80c2e in read_request_line (bb=0x7f4c6c67b7e8,
r=0x7f4c6c67a0a0) at /build/buildd/apache2-2.2.20/server/protocol.c:596
#21 ap_read_request (conn=0x7f4c6c6a92b8) at
#22 0x00007f4c7cd9a520 in ap_process_http_connection (c=0x7f4c6c6a92b8) at
#23 0x00007f4c7cd944b8 in ap_run_process_connection (c=0x7f4c6c6a92b8) at
#24 0x00007f4c7cda38ac in process_socket (bucket_alloc=<optimized out>,
my_thread_num=28, my_child_num=11, sock=0x7f4c6c6a90a0, p=<optimized out>)
    at /build/buildd/apache2-2.2.20/server/mpm/worker/worker.c:544
#25 worker_thread (thd=0x7f4c7cbfc980, dummy=<optimized out>) at
#26 0x00007f4c7c4beeb3 in ?? () from /usr/lib/
#27 0x00007f4c7c27befc in start_thread () from
#28 0x00007f4c7bfb5f8d in clone () from /lib/x86_64-linux-gnu/
#29 0x0000000000000000 in ?? ()

You are receiving this mail because:
You are the assignee for the bug.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message