httpd-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Merrell <ryanmerr...@gmail.com>
Subject [users@httpd] Single User DoS. How is this happening?
Date Tue, 24 Sep 2013 17:37:47 GMT
We've had a problem occur maybe 5 times over the last year where a
single user is DoS'ing our web server (unintentionally) and I'm having
a hard time figuring out how it's happening.


Background Info:

We're an online school with moderate traffic levels (800+ unique IPs
with 18,000 GET/POST over the last hour as of this post).
We're running Apache/2.2.15 Release 15.el6_2.1 from RHEL. We're
running RHEL 6.3.


The few times that this has happened, we've looked at the offending
user's traffic and noticed that before they DoS'd us, they were logged
in doing their course work as expected. But then we would suddenly get
thousands of GET requests for the home page, which showed up in the
access logs like this:

[18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496
[18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496
[18/Sep/2013:19:30:57 +0000] "GET / HTTP/1.1" 200 16496

In this last case, it was repeated over 7000 times in a span of about
18 minutes (generally around 10 requests per second). What ended up
happening is that we reached our MaxServerLimit number and Apache
eventually died. I've started looking at some tools, such as
mod_evasive, to protect us from such problems. But then I got thinking
about what was actually happening here, tried to reproduce the problem
myself, but couldn't! I used JMeter to simulate a heavy attack (more
connections at a faster rate -- around the ballpark of 25000 requests)
and I never spawned more than 15 child processes or so. My test
environment handled it perfectly. But something about how the student
did it made httpd spawn children like crazy, which eventually killed
it.

Here are some of my server configs that I feel are relevant. Any
advice on what is actually happening here and what I can do to
alleviate the problem would be appreciated.

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      2000
MaxClients       2000
MaxRequestsPerChild  4000
</IfModule>




 Ryan Merrell

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Mime
View raw message