[SURBL-Discuss] second and third level domains - again!

Eric Kolve ekolve at comcast.net
Sun Apr 25 11:50:08 CEST 2004

On Sun, Apr 25, 2004 at 05:57:56PM +0200, John Fawcett wrote:
> One of the things I noticed after upgrading to 
> SpamCopURI 0.14 was that previously I had
> been identifying all mail containing ads.msn.com
> as spam and after the upgrade this was no
> longer happening.
> (BTW ads.msn.com was still listed in sc.surbl.org
> when I observed this behaviour. It has since been
> removed.)

To take care of this in the short term you 
can add an entry to your spamcop_uri.cf if you 
have open redirect resolution on:

  open_redirect_list_spamcop_uri   g.msn.com             ads.msn.co

I have committed this change to the trunk, so you shouldn't
have to make the change by hand in the future.

The only way we can take care of this 100% of the time would be
to query for 2nd - 3rd for TLD and 3rd - 4th for ccTLDs.  This 
would catch subdomains if those were entered into the RBL.  
The downside is that this would increase the number of queries
two fold. We could reduce the queries if we stored a wildcard
A record for each the 2nd and 3rd level entries and store full entries
when needed.  Let me illustrate with the following examples:

RBL Hit - wildcard 3rd level
URL: http://www.foo.co.uk
DNS Query: www.foo.co.uk
Server Record: *.foo.co.uk

RBL Hit - 4th level
URL: http://ads.msn.co.uk
DNS Query: ads.foo.co.uk
Server Record: ads.foo.co.uk  # note fourth level domain

RBL Miss
URL: http://www.msn.com
DNS Query: www.msn.com
Server Record: n/a   
Result: n/a

RBL Hit - 3rd level
URL: http://ads.msn.com
DNS Query: ads.msn.com
Server Record: ads.msn.com  # note 3rd level

RBL Hit 2nd level wildcard
URL: http://bad.viagra.spammer.biz
DNS Query: viagra.spammer.biz
Server Record: *.spammer.biz

Basically this is client querying at the subdomain level.  
The biggest downside to a scheme such as this is that it would result
in less caching since there would inevitably be more unique queries.


> Version 0.14 includes the changes which were 
> being discussed last week, so that if ads.msn.com
> is found in an email only msn.com is being
> checked against sc.surbl.org.
> So the choices available to the list maintainer are either:
> - list all of msn.com
> - list none of msn.com
> Since listing all of msn.com is likely to be too wide, 
> this means msn.com will not get listed even if
> there are subdomains which are candidates for 
> listing.
> I've used msn as an example, but the same logic
> applies to any of the big names like yahoo etc where 
> the list maintainer may want to have more granularity
> in what is listed rather than list the whole registered 
> domain.
> The solution could be to use a special return code
> which indicates "query again with more detail".
> (I remember someone bringing up something similar
> in the context of ccTLDs as well).
> So if ads.msn.com were to be listed in sc.surbl.org
> it would need two records:
> msn.com IN A
> ads.msn.com IN A
> The client (in this case SpamCopURI) would
> find a url ads.msn.com in the email but would
> query for msn.com as per the current logic.
> The return value of then indicates
> to the client to query for one level 
> lower, ie ads.msn.com.
> This same mechanism could be used for ccTLDs.
> sc.surbl.org could contain:
> co.uk IN A
> co.nz IN A
> So that if I get xxxxxxxx.co.uk in an email, 
> the client queries for co.uk and it will be told 
> to query with the lower level. The client
> queries for xxxxxxxx.co.uk
> The disadvantage of this solution is that it needs
> a changed logic on the client side, however the
> client does not need to know anything about
> which ccTLDs have two or three level domains.
> The extra query overhead is minimal, since
> on a busy server these additional records would
> almost always be in the DNS resolver cache.
> Even clients which don't implement this extra 
> logic would however have to be careful to check
> the value of the A record rather than deduce the 
> listing from the presence of an A record.
> Maybe we're still in time to make changes
> like this before there is too much software 
> to change (currently I think there are 3).
> John
> _______________________________________________
> Discuss mailing list
> Discuss at lists.surbl.org
> http://lists.surbl.org/mailman/listinfo/discuss

More information about the Discuss mailing list