[SURBL-Discuss] Re: second and third level domains - again!

John Fawcett johnml at michaweb.net
Tue Apr 27 23:47:41 CEST 2004

Jeff Chan wrote on Mon Apr 26 03:58:58 CEST 2004 

> The underlying principle as I see it is that most
> major site will have functional anti-abuse and anti-spam
> policies, so either a base domain is good or bad.  I
> know that seems simplistic, but it's easy and fast to
> implement AND it seems to match reality pretty well.

There may come a point where a "big" domain 
starts to appear in spam despite an outward
image of "antispam" and the choice which the
current infrastructure offers is to block all of it or 
none of it. Blocking all of it may produce
too many FPs whereas blocking a subset may be
acceptable. At the moment there haven't
been any tough decisions  to take on listing or
whitelisting. Everything has been very clear cut.
It's not guaranteed to stay that way.

I don't know how real a risk this is, but I was
worried about an infrastructure which effectively 
ties our hands on this point. (Once surbl is 
deployed in many different client software, 
I suppose it will be hard to change anything about
the "public interface" to the data....)

> That's an interesting idea.  Basically you want to
> signal redirection to higher domain levels with a
> special result for levels that should never get
> checked like co.uk.

> That might be doable, but it would require extra
> logic on the client side as you note.  That already
> sounds more complex than I like, though I see what
> you're getting at.  Better to control what goes into
> the data (i.e. never let the TLD itself co.uk in),
> and make sure the client is following similar rules.

What I was proposing is a *change* to client
processing logic, but not added *complexity*.
rather a simplification. 

What I mean is that currently the client has to
- processing logic
- data on the ccTLDs 

(in the long run there will be multiple versions 
of the ccTLD data implemented in various client 
versions at any one time. Users will have to 
upgrade the client software to keep current
with domain data).

Using an A record in surbl to indicate 
"this domain is not listed but a subdomain is. Try again"
would mean the client just follows a simple
processing rule. It doesn't need to know 
anything about specific domain data.

The logic would be something like this.
lets assume that "url" is found in an email.

more_info_result = ''
listed_result = ''
    result = query_dns ( extract_domain (url, level))
    level = level +1 
}while (result == more_info_result)

if (result == listed_result) { score it }

> We will always catch bigspammer.co.uk with the current
> strategy.

I think that some of the ccTLDs have a mixed assignment
strategy. This means that they should sometimes be
checked at the 2nd level sometime at the 3rd level. 
The current logic always checks at a single predefined level.

The two example I saw were: .fr and .ca

Currently we check .ca at the third level, but it is possible
to register a second level domain at .ca which we never catch
so bigspammer.ca will get through.

We check .fr at the second level however there are 
many "standard" second level domains (like .nom.fr) which 
means we probably want to be checking these ones at the third 
level. (Translation: any bigspammer.nom.fr domain is imune to
the current strategy unless we want to upset everyone who has
a nom.fr domain by listing that).

> I know a lot of what I argue for above seems simplistic
> when a more complex solution could have more interesting
> results, but very often the simpler solutions are better,
> especially in terms of resource consumption.

You're right to argue for the most simple solution. (In fact
there are probably simpler solutions than the one I
suggested!). My concern was to avoid inflexibility
in the infrastructure and automatic immunity for various 
classes of domains. 


More information about the Discuss mailing list