spamassassin-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Kettler <mkettler...@verizon.net>
Subject Re: How do I get better processing/delivery times?
Date Thu, 10 Jun 2010 11:24:07 GMT
On 6/9/2010 7:51 PM, Spiro Harvey wrote:
> I maintain a mail cluster that gets about 70,000 messages a day per
> node.
>
> I'm just wondering if it's possible to decrease the scan times. In the
> TOTALS section AvgTm is the average "scantime" in the spamassassin log
> file:
>
> (Delivered are messages that SA scores under 5, Spamboxed are scored
> 5+, but under 10, and Rejected are 10+)
>
>
>
> # ./knl-spam-stats.awk /var/log/spamassassin.*
>
>  TOTALS                                              
>                               AvgTm  AvgThruput
>              # Msgs   %/Total (sec) (bytes/sec)
>              ~~~~~~ ~~~~~~~~~ ~~~~~ ~~~~~~~~~~~
>   Delivered  176086 ( 17.80%) 15.22     2208.40
>   Spamboxed   51194 (  5.17%) 19.92      550.14
>   Rejected   762189 ( 77.03%) 19.30      537.56
>
>   Total      989469 messages processed
>
>   (70676/day; 2944.85/hr; 49.08/min; 0.82/sec)
>
>  BLACKLIST HITS                                       
>   Blacklist       Msgs   %/Total   %/Spam Avg Score
>   ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~ ~~~~~~~~~
>   Spamhaus SBL    2087 (  0.21%) (  0.26%)    16.15
>   Spamhaus PBL  569825 ( 57.59%) ( 70.06%)    21.99
>   Spamhaus XBL  497403 ( 50.27%) ( 61.15%)    21.98
>   SBL URI       187292 ( 18.93%) ( 23.03%)    26.05
>   NJABL           3544 (  0.36%) (  0.44%)    23.65
>   SORBS         387539 ( 39.17%) ( 47.65%)    22.32
>   Spamcop       513748 ( 51.92%) ( 63.16%)    22.48
>   SURBL URI     360620 ( 36.45%) ( 44.34%)    27.25
>   RFC Ignorant   29295 (  2.96%) (  3.60%)    20.68
>
>  CUSTOM RULE HITS                                     
>   Custom Rule     Msgs   %/Total Avg Score
>   ~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~ ~~~~~~~~~
>   MIME/JPG          84 (  0.01%)    15.45
>   ZIP file         741 (  0.07%)    18.44
>
>
> Yet, on another mail cluster that only gets 4-5000 messages a day per
> node, the average scantimes are 4-5 seconds. Both have the same custom
> rules, so any slowness in processing regexes should be noticable on
> both systems.
>
> In the first case, we have started rsyncing Spamhaus' blacklists in the
> hopes that it would increase scantimes by decreasing DNS lookup times.
> It hasn't really made too much difference, but my main concern is that
> the messages seem to be taking so long regardless.
>
> The boxes are running Sendmail 8.14 + ClamAV 0.96 + SA 3.3.1 + Razor
>
>
> SPAMDOPTIONS="-d -x -c -m50 -H -s local2 /home/spamd -u spamd
> --min-children=10 --min-spare=10"
>
> Core 2 Duo @2.93GHz, 4GB RAM. Load averages typically sit at 5-7 during
> the day.
>
> Any advice on how I can tune the scantimes?
>
>
>   
These settings:
-m 50 --min-children=10 --min-spare=10

seem a bit high for a box with only 4GB of ram... Is the box suffering
from severe swap usage, and grinding itself to a halt when all 50 are up
and running? (try running "free", what does it say?)


I might suggest something more like 10-20 as a max children with 4gb of ram:

-m 10 --min-children 5 --min-spare=1
-m 20 --min-children 10 --min-spare=2

Adding more children helps, but only if you have enough ram to fit them
all. Once you run out of ram, performance suffers severely.

Or, as the manpage for -m says:

Note that if you run too many servers for the amount of free RAM
available, you run the danger of hurting performance by causing a high
swap load as server processes are swapped in and out continually.


Mime
View raw message