spamassassin-sysadmins mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Jones <da...@apache.org>
Subject Re: sa-vm1.apache.org is unresponsive
Date Sun, 07 Jan 2018 14:08:15 GMT
I believe the cause of this problem was bad bots ignoring the robots.txt 
excessively hitting the ruleqa.cgi which takes a second or two to run 
eating up memory and CPU.

I have setup this bad bot blocker to prevent this from happening again:

https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker

Let's keep an eye on things for a bit and see if this helps keep this 
server from falling over again.

Dave


On 01/05/2018 09:17 AM, Dave Jones wrote:
> On 01/04/2018 08:07 PM, Kevin A. McGrail wrote:
>> On 1/4/2018 9:03 PM, Dave Jones wrote:
>>> VM good now.  Chris bumped it up to 16 GB.  I am not complaining but 
>>> I think that something is going wrong that will eat up all the RAM no 
>>> matter how much there is.
>> I am really not used to unstable Linux boxes no matter the flavor. 
>> This is, however, the only single one I have without swap.
> 
> The OOM killer started killing processes so it was out of memory.  There 
> were a lot of apache2 and ruleqa.cgi processes.
> 
> The Apache logs show a lot of bots so I will setup a robots.txt to keep 
> them from following all of the links of the page for no good reason.
> 
> The last OOM killer problem started at 1/4/18 11:41 UTC and at that time 
> the Apache logs show several bots hitting the server at once.  Hopefully 
> the robots.txt will solve our problem but we may need to enable Apache 
> max connections to prevent too many ruleqa.cgi's from launching at the 
> same time.  Someone could easily open a few dozen/hundred connections 
> and DOS the server.
> 
> Dave



Mime
View raw message