httpd-bugs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 61551] New: Event MPM workers stuck in Gracefully Finishing with no connections left
Date Wed, 20 Sep 2017 16:30:49 GMT
https://bz.apache.org/bugzilla/show_bug.cgi?id=61551

            Bug ID: 61551
           Summary: Event MPM workers stuck in Gracefully Finishing with
                    no connections left
           Product: Apache httpd-2
           Version: 2.4.27
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mpm_event
          Assignee: bugs@httpd.apache.org
          Reporter: nic@nicj.net
  Target Milestone: ---

I've been having an issue with Apache's event MPM and Slots being taken up by
PIDs that never seem to exit the "yes (old gen)" ("Gracefully finishing")
state.

I first thought this was related to
https://bz.apache.org/bugzilla/show_bug.cgi?id=53555, but that was fixed in
2.4.24, and I'm running 2.4.27.

The problem started 3 days ago when I switched from prefork to event MPM. 
Since then, I've had 3 occurrences of Apache stopping accepting any new
connections.  After a few minutes, my monitoring scripts detect this and
restart httpd.

I'm still debugging exactly what the state of Apache is at that time, but I see
this in the event log:

AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.

After reading bug # 53555, I've tweaked my server settings a little bit
(specifically, increased ServerLimit to be more than
MaxRequestWorkers/ThreadsPerChild so there is Scoreboard room available after a
reload when some slots are taken by shutting-down processes).

I'm not sure if that will fix the issue of server-not-responding long-term, but
I've noticed that Apache is often getting into a state where there are
processes stopping, but stuck in "yes (old gen)" state indefinitely.

These processes don't appear to ever exit, until I hard restart Apache.

These "yes (old gen)" processes are Accepting Connections=no, and 0 Threads
busy/idle, and Async connections writing/keep-alive/closing are all 0 too.

An example scoreboard is below:

        PID     Stopping        Connections     Threads Async connections
total   accepting       busy    idle    writing keep-alive      closing
0       19022   no      29      yes     5       59      0       18      8
1       5502    yes (old gen)   44      no      0       0       0       0      
0
2       8233    no      5       yes     0       64      0       4       1
3       5557    yes (old gen)   19      no      0       0       0       0      
0
4       5597    yes (old gen)   19      no      0       0       0       0      
0
Sum     5       3       116             5       123     0       22      9

You can see there are 3 processes that are "yes (old gen)" with 0 active things
going on.

The scoreboard shows many things in the "G" state:

_______W_____W___________R_____________________________WW_______
G...G.GGGGGGG.GGGGGGG..GGGG.G.G.G..GGG.GGGGGGGGG..GGGG.G.GG...GG
________________________________________________________________
G.....G...GG.......GG..G...G.GG......G.GGG...G...GG...G.......G.
.....G.GGG...GGG.....GG....G..G.GGG......GG.......G........G...G
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................

I'm not sure if there's a configuration setting I'm missing to ensure those
"yes (old gen)" processes actually exit earlier, or if this is a bug.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


Mime
View raw message