httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 56034] New: CPU consumption due to circular linked list in ap_core_input_filter()
Date Mon, 20 Jan 2014 03:51:45 GMT
https://issues.apache.org/bugzilla/show_bug.cgi?id=56034

            Bug ID: 56034
           Summary: CPU consumption due to circular linked list in
                    ap_core_input_filter()
           Product: Apache httpd-2
           Version: 2.2.15
          Hardware: HP
                OS: HP-UX
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Core
          Assignee: bugs@httpd.apache.org
          Reporter: venkatunix02@gmail.com

We run Apache 2.2.15(+ fixes for certain CVEs) on HP Itanium server with HP-UX. 

Once in a while, we observe ~100% CPU consumption by the httpd processes.


1. configuration

MPM worker configuration:
<IfModule mpm_worker_module>
    ServerLimit          80
    StartServers         67
    MaxClients          4000
    MinSpareThreads     3350
    MaxSpareThreads     4000
    ThreadsPerChild     50
    MaxRequestsPerChild  1200
</IfModule>


web server: Reverse proxy

3rd party module: Oracle webgate


2. Logs

Please find the attached files:
a) stack trace : bt_all
b) top and syscall trace (top_syscall_trace) 
c) Mapping between stack trace, source code and itanium assembly
code(trace_srccode_assemblyCode)

3. Trace identified from the core dump generated in that scenario:

syscall trace shows that main thread waits for the user space thread that run
ap_core_input_filter()

trace:
Thread 22 (system thread 1947121):
#0  0x40000000000977f0:2 in ap_core_input_filter () at core_filters.c:136
#1  0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489
#2  0xc000000000ac76c0:0 in logio_in_filter () at mod_logio.c:129
#3  0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489
#4  0x4000000000074980:0 in ap_rgetline_core () at protocol.c:231
#5  0xc000000004a33580:0 in ap_proxygetline () at mod_proxy_http.c:1658
#6  0xc000000004a338c0:0 in ap_proxy_http_process_response () at
mod_proxy_http.c:1733
#7  0xc000000004a35ee0:0 in proxy_http_handler () at mod_proxy_http.c:2434
#8  0xc000000004929a10:0 in proxy_run_scheme_handler () at mod_proxy.c:2675
#9  0xc000000004928b60:0 in proxy_handler () at mod_proxy.c:1024
#10 0x400000000009d2d0:0 in ap_run_handler ()
#11 0x400000000009e9e0:0 in ap_invoke_handler () at config.c:381
#12 0x40000000000cdec0:0 in ap_process_request () at http_request.c:282
#13 0x40000000000c5a40:0 in ap_process_http_connection () at http_core.c:190
#14 0x40000000000b8d20:0 in ap_process_connection () at connection.c:189
#15 0x40000000000de4c0:0 in process_socket () at worker.c:590
#16 0x40000000000df860:0 in worker_thread () at worker.c:974
#17 0xc000000006f95200:0 in dummy_worker () at threadproc/unix/thread.c:160
#18 0xc00000000013fb20:0 in __pthread_bound_body () at pthreads/pthread.c:4875

Eventhough it shows Line:136, actual loop happens @ line 141:
   140      /* ### This is bad. */
   141      BRIGADE_NORMALIZE(ctx->b);

Due to compiler optimzation and macro @ line 141, this line no is not shown in
trace.


As shown in the trace_srccode_assemblyCode attachment, linked list has become
circular because address and content of (ctx->b)->list))->next are same

(gdb) info register r44
   gr44: 0x6000000000777de8

(gdb) x/gx0x6000000000777de8
0x6000000000777de8:     0x6000000000777de8

Prev’s value also contains the same.
(gdb) x/gx 0x6000000000774508
0x6000000000774508:     0x6000000000777de8


Please help to resolve this.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message