Mailing-List: contact dev-help@spamassassin.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: neutral (hermes.apache.org: local policy)
Date: Sat, 4 Sep 2004 10:45:44 -0600 (CST)
From: Ryan Thompson <ryan@sasknow.com>
To: Jeff Chan <jeffc@surbl.org>,
        SURBL Discussion list <discuss@lists.surbl.org>
cc: SpamAssassin Developers <spamassassin-dev@incubator.apache.org>
Subject: Re: [SURBL-Discuss] 	checking plain domains in message bodies against
 SURBLs reportedly	effective
In-Reply-To: <1411296005.20040904042613@supranet.net>
Message-ID: <20040904102324.O53861@drizzle.sasknow.net>
References: <1411296005.20040904042613@supranet.net>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

Jeff Chan wrote to SpamAssassin Developers:

> Randy Brukardt of rrsoftware.com mentioned that checking
> plain domains occurring in message bodies against SURBLs
> was pretty productive.  (E.g., look for domain.com in
> addition to www.domain.com or http://www.domain.com).
>
> Perhaps this could be something interesting to at least try
> experimentally or to think about.

Yep. Good idea, overall. There are a few gotchas:

TLD extensions sometimes map file extensions. We might have to whitelist
command.com, and the entire country of Poland. :-)

Looking at the above sentence, leading/trailing punctuation might be a
potential snag.  I.e.: 4 cheap pillz, go to somethingsleazy.com, and
give us your money.

Since the domain is in plain text and doesn't contain a protocol or
subdomain (i.e., 'www'), I haven't yet seen a mail client that will
display it as a clickable URL. Thus, with this, we're probably mostly
fighting the "type this in" or "cut and paste into your browser" type of
spammer. SO, if we do this, implementers could force spammers to
obfuscate the domains beyond recognition. They'll have to do their own
munging, and we might try to catch it, but that's risky. "i looked on
the boss' computer and found porn. info forthcoming...", or even,
"spammer dot com operations are a plague on civilized nations".

Any implementations will probably have to run against large ham corpora
to see if anything like the above becomes falsely *extracted* as a URI,
regardless of whether the current data happens to cause a FP.

I'd advise keeping implementations simple and strict by default (i.e.,
no deobfuscation; maybe just clickable links only), and allow the user
to control the amount of fuzziness they'd like to match on.

- Ryan

-- 
   Ryan Thompson <ryan@sasknow.com>

   SaskNow Technologies - http://www.sasknow.com
   901-1st Avenue North - Saskatoon, SK - S7K 1Y4

         Tel: 306-664-3600   Fax: 306-244-7037   Saskatoon
   Toll-Free: 877-727-5669     (877-SASKNOW)     North America