[SURBL-Discuss] ccTLDs and multiple queries

Eric Kolve ekolve at comcast.net
Wed Apr 21 16:12:58 CEST 2004


On Wed, Apr 21, 2004 at 03:00:52PM -0700, Jeff Chan wrote:
> On Wednesday, April 21, 2004, 12:21:16 PM, John Fawcett wrote:
> > From: "Eric Kolve"
> >> Initially, when I released spamcopuri I decided to pretty much ignore
> >> whether the TLD was a country code or not.  This results in about
> >> twice as many queries as necessary, but guaranteed you would get
> >> hits if the domain was listed.
> >>
> >> Now that people are pointing this to other RBL's beside just surbl,
> >> should we continue to do second and third level queries? Or just
> >> the query that we assume to be necessary?  My concern is that not
> >> all RBLs will process the domains according to a list such as
> >> http://www.bestregistrar.com/help/ccTLD.htm.  I suppose the worst
> >> case scenario is we end up getting a miss when we should be getting
> >> a hit because one side presumes that say TLD .za has a subdomain 'foo',
> >> when the server doesn't.  The server side would expect a second level,
> > while
> >> the client would do a third level query (this is why I wanted the wildcard
> >> records).  I guess this really isn't that great a consequence considering
> >> the savings and the fact that this shouldn't occur very often.
> >>
> >> I will go ahead and make this change if everyone is comfortable with the
> >> known risk.
> >>
> > I think if an rhsbl is listing a second level registry domain
> > (like .co.uk) then I think it's up to the list maintainer to implement
> > the wild card so that xxxxx.co.uk returns an A record. I wouldn't
> > worry about taking into account such an extreme case,
> > since I cannot imagine any list wanting to do such widespread
> > blocking.
> 
> Yes, the two level ccTDLs like co.uk should never get into a
> SURBL.  Only registrar-type domains should, like foo.co.uk.
> 
> > I believe there should be a mechanism which distinguishes whether
> > a second or third level lookup is required based on a static
> > lists of domains known to have or not have subdomains.
> > If nothing is known then the default should be to check both
> > second and third as at present.
> 
> Aha, now I think I understand what's being proposed.
> 
> Currently SpamCopURI checks all domains at the second
> and third level against a given SURBL, regardless of
> whether the domain is in a ccTLD or not.
> 
> It sounds like Eric is proposing a change, where if a domain is
> in the ccTLD list like co.uk, then the client should try
> extract and check a three level domain like foo.co.uk.  Otherwise
> it should check two levels like foo.com.
> 
> Is that right?  If so it may be ok, though our list of ccTLDs is
> slightly underspecified (there are some ccTLDs not in it).  Note
> that my ccTLD list:

Yes.  This is exactly what I am proposing.

> 
>   http://spamcheck.freeapp.net/two-level-tlds
> 
> is (derived from but) slightly more complete than the one at
> http://www.bestregistrar.com/help/ccTLD.htm ....
> 
> Worst case is that we miss a few ccTLDs.  Probably not too big a
> deal given that most of the spam domains are .com, .biz, etc.
> 
> I believe Eric is also making a finer point that other SURBL data
> sources may miss some unexpected geographic domains where foo.za
> occurred and only two-level base-ccTLDs like foo.com.za were
> expected. Not sure how to handle unusual cases like that.  I
> suppose we'll need to relay on the country code authorities to be
> somewhat consistent with respect to what levels they will allow
> in their ccTLD.
> 
> Philosophical point: it's always possible that some spam domains
> slip through the cracks, but if that happens often enough and
> we spot them, we can always blacklist them manually.  Perfection
> may not be possible, but we're certainly greatly increasing the
> spam detection rates with this approach overall.

My only concern is that we leave a wide enough of a hole that
we end of playing catch-up and spammers run through various ccTLDs
that we have mis-classified using them for links.

> 
> BTW I'm using the ccTLD list to try to ensure that any two level
> ccTLDs do *not* get into any SURBLs.
> 
> P.S. ***If anyone has more-complete ccTLD lists or any updates or
> additions, please share them,*** else spammers may set up shop in
> some unknown Outer Mongolian ccTLD, and we may not catch them. :-)
> 
> Jeff C.
> 
> _______________________________________________
> Discuss mailing list
> Discuss at lists.surbl.org
> http://lists.surbl.org/mailman/listinfo/discuss


More information about the Discuss mailing list