Mailing-List: contact users-help@spamassassin.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (nike.apache.org: domain of mkitchin.public@gmail.com
 designates 209.85.160.178 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=Q46dXPuauaGWbIoJ+vz8vfPqsr2anMmuEuB3mUntoNfYJyV26P1gpGPViyYkAi+fqy
         BUzCTnHW3PZKI3eDeTwXfMo7JlvnQBimXHT6Tu9ZbPyDV+VhsXUjatBUK27kH/UR9qKH
         MIyoYWz7bRRt8mP/B06n/lE9pBTgJ5rH2cMcc=
Message-ID: <4C600B55.2030603@gmail.com>
Date: Mon, 09 Aug 2010 09:06:13 -0500
From: "Matthew Kitchin (public/usenet)" <mkitchin.public@gmail.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
 rv:1.9.2.8) Gecko/20100802 Thunderbird/3.1.2
MIME-Version: 1.0
To: users@spamassassin.apache.org
Subject: Re: List of "banned" words/bounce to sender
References: <1281355094.2104.30.camel@zappa.gregorie.org>
 <C8855EAA.12165%dan.mcdonald@austinenergy.com>
 <20100809132758.GA14804@smtp.hege.li>
In-Reply-To: <20100809132758.GA14804@smtp.hege.li>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

  On 8/9/2010 8:27 AM, Henrik K wrote:
> Nope, people constantly underestimate the power of regexes.. of course you
> can easily make bad ones, but Perl can run huge lists of simple alternations
> FAST.
>
> I downloaded a 10000 random name pack, and made a quick hack to regexify it
> with my favourite Regexp::Assemble.
>
> ------------------------------
> #!/usr/bin/perl
> use Regexp::Assemble;
> $ra = Regexp::Assemble->new;
> while (<STDIN>) {
>      chomp;
>      # Read comma separated names from stdin: Firstname,Lastname
>      ($firstname, $lastname) = split(',', lc);
>      # Firstname Lastname
>      $ra->add("$firstname $lastname");
>      # Lastname,? Firstname
>      $ra->add("$lastname,? $firstname");
>      # Print rule every 10000 names
>      # (?:^| ) instead of \b since "Kate" would hit "Mary-Kate"
>      if (++$cnt % 10000 == 0 || eof STDIN) {
> 	print 'body TEST_NAMES_'.++$idx;
>          print ' /(?:^| )'.$ra->as_string.'(?:$| )/i'."\n";
>      }
> }
> ------------------------------
> ./names.pl<  names.csv>  names.cf
>
> The resulting single 170000 byte rule did not affect SA in anyway, there was
> virtually no difference in my mass check tests. Running the regex through
> some file manually results in 80000 lines/second. This with one 3Ghz core.
> I think you can make rules/REs of MBs in size, but gains probably nothing.
>
> About ClamAV...
>
> + It would probably handle this even faster
> + Easy logging of exact signature that got hit (single name per sig)
> - It would also match any header like To: From: etc (PRETTY BAD...)
>
> I'd choose SA since it's way more flexible. I doubt performance here is a
> factor, especially with outgoing mail..
>
Thanks for the info.

- It would also match any header like To: From: etc (PRETTY BAD...)

That could be an issue. I will check to see if I can find a workaround, 
if not, ClamAV may not be an option.