spamassassin-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Wolfe" <aawo...@gmail.com>
Subject Re: Handling Spam Surges
Date Mon, 10 Sep 2007 18:51:03 GMT
On 9/10/07, Paul Griffith <paulg@cse.yorku.ca> wrote:
>
> Greetings,
>
> How do you handle Spam surges/DoS attacks? We just had a Spam surge/DoS
> and are looking at ways to better withstand (as best as we can) another
> surge
>
>
> Here is how we start SA:
>
> -c -d -r $PIDFILE -s /var/log/spamd --socketpath=$SOCKET
> --max-children=150 --min-children=10
>
> Our (1) mail server is configured like this:
>
> CentOS 4.5
> Exim 4.67
> SpamAssassin version 3.2.3 running on Perl version 5.8.8
> ClamAV 0.91.2 (saneSecurity updates)
> - handles incoming/outgoing mail
> - handles imap/pop/webmail request
>
> Intel D Cpu 3.00Ghz with 2GB of Mem
> 80GB SATA root disk
> 200GB SATA mail disk (softraid mirror)
> 2xIntel e1000
>
> Our mail server was taking a pounding on Friday,
>
> Fri Sep  7 16:17:09 2007 [26914] info: prefork: child states: BBBBB
> Fri Sep  7 16:17:09 2007 [26914] info: prefork: child states: BBBBBB
> Fri Sep  7 16:17:09 2007 [26914] info: prefork: child states: BBBBBBB
> ..snip...
> ..snip...
> ..snip...
> Fri Sep  7 16:17:17 2007 [26914] info: prefork: child states:
> BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
> Fri Sep  7 16:17:17 2007 [26914] info: prefork: child states:
> BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBIBBB
> Fri Sep  7 16:17:19 2007 [26914] info: prefork: child states:
> BBBBBBBBBBBBBBBBBBBBBBBBBIBBBBBBBBBBBBBBBBBBBBBBIBBB
> ..snip..
> ..snip..
> Fri Sep  7 16:19:22 2007 [26914] info: prefork: child states:
>
> BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBSBBSBB
> Fri Sep  7 16:19:23 2007 [26914] info: prefork: child states:
>
> BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBIBBISBB
>
> At the mist of the surge we had 95 child processess running, all busy!
>
> Here are the sar memory stats...
>
>                kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree
> kbswpused  %swpused  kbswpcad
> 16:10:02        16804   2056424     99.19      2900   1310880
> 2040036       208      0.01         0
> 16:20:10        37676   2035552     98.18      1872    237376   1736152
> 304092     14.90     78992
> 16:30:51        13924   2059304     99.33      1292    308944   1044160
> 996084     48.82    357444
> 16:40:02        76652   1996576     96.30      8208   1280796   1756236
> 284008     13.92    178696
> Average:        26403   2046825     98.73      5880   1364057
> 2024199     16045      0.79      6152
>
>
> Here are the warnings we saw in the spamd log...
>
> Fri Sep  7 16:20:39 2007 [26914] info: prefork: child states:
>
> IBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
> Fri Sep  7 16:20:40 2007 [25431] warn: spf: lookup failed: Can't locate
> object method "new" via package "Net::DNS::RR::TXT" at
> /xsys/lib/perl5/site_perl/5.8.8/i686-linux/Net/
> DNS/RR.pm line 312.
> Fri Sep  7 16:20:41 2007 [25428] warn: spf: lookup failed: Can't locate
> object method "new" via package "Net::DNS::RR::TXT" at
> /xsys/lib/perl5/site_perl/5.8.8/i686-linux/Net/
> DNS/RR.pm line 312.
>
> Fri Sep  7 16:22:18 2007 [24684] warn: plugin: eval failed: child
> processing timeout at /xsys/sbin//spamd line 1246, <GEN683> line 3398.
> Fri Sep  7 16:22:20 2007 [24711] warn: Use of uninitialized value in
> pattern match (m//) at /xsys/lib/perl5/5.8.8/utf8_heavy.pl line 211,
> <GEN749> line 3398.
> Fri Sep  7 16:22:20 2007 [24711] warn: Use of uninitialized value in
> scalar assignment at /xsys/lib/perl5/5.8.8/utf8_heavy.pl line 227,
> <GEN749> line 3398.
>
> Fri Sep  7 16:26:15 2007 [25227] info: spamd: clean message (1.5/5.0) for
> cs242027:9190 in 406.1 seconds, 243776 bytes.
> Fri Sep  7 16:26:19 2007 [24688] warn: spamd: copy_config timeout,
> respawning child process after 1 messages at /xsys/sbin//spamd line 1103.
> Fri Sep  7 16:26:24 2007 [25046] warn: spamd: copy_config timeout,
> respawning child process after 1 messages at /xsys/sbin//spamd line 1103.
> Fri Sep  7 16:26:28 2007 [24692] warn: spamd: copy_config timeout,
> respawning child process after 1 messages at /xsys/sbin//spamd line 1103.
>
> Fri Sep  7 16:30:35 2007 [26914] info: spamd: server successfully spawned
> child process, pid 26312
> Fri Sep  7 16:30:37 2007 [24685] warn: spamd: copy_config timeout,
> respawning child process after 1 messages at /xsys/sbin//spamd line 1103.
> Fri Sep  7 16:30:39 2007 [26914] warn: prefork: cannot ping 24702, file
> handle not defined, child likely to still be processing SIGCHLD handler
> after killing itself
> Fri Sep  7 16:30:39 2007 [26914] warn: prefork: killing failed child 24702
> fd=undefined at
> /xsys/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/SpamdForkScaling.pm line
> 171.
> Fri Sep  7 16:30:39 2007 [26914] warn: prefork: killed child 24702
> Fri Sep  7 16:30:41 2007 [26914] info: spamd: handled cleanup of child pid
> 24702 due to SIGCHLD
> Fri Sep  7 16:30:41 2007 [26914] warn: prefork: cannot ping 24687, file
> handle not defined, child likely to still be processing SIGCHLD handler
> after killing itself
> Fri Sep  7 16:30:41 2007 [26914] warn: prefork: killing failed child 24687
> fd=undefined at
> /xsys/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/SpamdForkScaling.pm line
> 171.
> Fri Sep  7 16:30:41 2007 [26914] warn: prefork: killed child 24687
>
>
> Looking at the swap usage, I was thinking I would be better if I reduced
> the number of children processes and let thing queue up. I know I will
> also have to look at exim and it's ratelimit command. Any other idea's on
> handling spam surges/DoS?
>
> Thanks
> Paul
>


At my site we operate under the presumption that SpamAssassin should be
avoided if at all possible because it is so expensive on our resources
compared to some other easy checks.  This helps us to deal with DoS and
"surges" from retarded bots quite well (so far at least).

We reduce the messages bound for SA to less than 10% of our traffic by a
combination of postfix UCE checks, a couple very accurate RBLs, selective
greylisting and our own whitelist.  When the surges/DOS happen, they tend to
increase the number of messages thrown away but rarely effect the volume
running through SA.

-Aaron

Mime
View raw message