[SPAM-TAG] [SURBL-Discuss] Subdomains in SURBL
jeffc at surbl.org
Fri May 12 17:48:38 CEST 2006
On Friday, May 12, 2006, 7:59:57 AM, Brandon Hutchinson wrote:
> Looking at the multi.surbl.org zone yesterday, I noticed approximately 373
> subdomains in the list.
> Here are a few examples:
> Looking at http://www.surbl.org/implementation.html item 2, do these
> subdomains belong in the list?
> "Extract base (registrar) domains from those URIs. This includes removing any
> and all leading host names, subdomains, www., randomized subdomains, etc. In
> order to determine the base domain it may be necessary to use a table of
> country code TLDs (ccTLDs) such as this partially-complete one SURBL uses.
> (Note that this file is only rarely updated. Please don't download it
> frequently.) For example, any domain found in the two level ccTLD list should
> have a three-level domain name extracted (like foo.co.uk) for matching
> against a SURBL. Domains not specifically on the two level ccTLD list (such
> as foo.com or foo.fr) should be checked at two levels."
Most of the listed records with subdomains deeper than we'd
normally list are from phishes. It's true that they don't follow
the specification, but they're exceptional. Most of the domains
*are* reduced to registered levels on the data side, where it's
clear the domains belong to the phishers or spammers.
> I believe SpamAssassin's URIDNSBL reduces the URIs to the base domain (e.g.
> example.com, example.co.uk), so if it encountered "www.freecat.biz," for
> example, it would lookup freecat.biz, which is not in the list.
That's correct. It may check other levels too, but the spec says
to check GTLDs at the second level and CCTLDs in the table at the
third. There may be other outlying cases in terms of the number
of levels that should be checked, but two and three levels of
GTLDs and CCTLDs certainly covers most of the common spams.
> Besides URIDNSBL, are there other URI lookup implementations for which it
> makes sense to include subdomains?
Not sure I understand the question. Can you elaborate?
It may help to know what problem you're trying to solve.
Don't harm innocent bystanders.
More information about the Discuss