...
JeffC of SURBL asked:
It has been suggested that we could deal with the tripod.com subdomains by adding tripod.com to our two-level-tld list or some equivalent file. Currently the two-level-tld list is hard coded into applications to indicate that domains like co.uk should be checked at the third level, like checkmehere.co.uk. This is somewhat of a kludge, but it works.
How does anyone feel about extending that to tripod.com and potentially other hosts that provide subdomain hosting?
If so, should it be done in the same two-level-tld file or maybe a different, separate file? (Currently the two-level-tld file is mostly geographic domains, i.e. cctlds plus the local country registrar top level domains.) Or should it be done with another, separate DNSBL, etc. Both approaches have advantages and disadvantages. The file is much simpler to maintain for us, but a bit kludgey.
The hard-coding into applications is just plain wrong for a few reasons; Any list *should* be updatable. Some RHS BLs already support arbitrary subdomains (example: biz.mail.mud.yahoo.com has many more problems than yahoo.com or most yahoo.com subdomains). For DSN, RCVD and other header rules, the "top" domain, (e.g. "ca.us") may not have, or even be required to have an abuse and/or postmaster account (just one possible type of example), but the subdomains either do have them or would trigger tests or rules for them (a domain with no 'A' or 'MX' records may not need to provide any mail services, though its subdomains should and do - your losangeles.ca.us vs. "ca.us" example). Also note that some MTAs (eg. Postfix) already check *all* of the subdomains in a hostname against RHS RBLs.
One problem with this is the syntax for DNS "wildcards" transcends subdomain levels, so I believe the files/domains should be kept separate from the "true" two-level TLDs, because unfortunately a check for '*.domain.tld' should really be performed also (e.g. '*.domain.tld' in a DNS zone file will match 'l5.l4.l3.domain.tld' as well as the simple 'l3.domain.tld') and this leads to increased overhead for DNS based net tests (without the check, the containing domain cannot be determined by heuristics alone).
One set of candidates that would also fall into this group include ESPs like cheetah/chtah (discussed a few months ago), where most all (or at least many) of the subdomains are well behaved, but some are definitely "mainsleaze-like" or just plain spammy (not meant to reflect on cheetah specifically here). Similarly, some poorly behaved companies are kind enough to segregate their marketing "spew" into a subdomain, allowing just that to be listed *and* tested for (think of more than just the SURBL lists - but anything SA or other apps may want to check).
Conceivably the file or RBL could be used for any arbitrary number of levels, for example if subdomains like spammerz.losangeles.ca.us (unlikely example, but you get the idea) or something similar started appearing in spams.
The use of most geographic domains in the body of spam is still rare, but I know that you have seen examples of them used for both dropboxes and domain contacts (and some geographic domains, at least in the ".us" TLD have been suspended for just this cause - directly or indirectly).
If we make the change, we'd need to let the SURBL application authors know to update their tld file, etc., and we'd also need to update our data-side processing to allow subdomains to be listed.
Comments?
The data processing changes needed for applications are the hardest ones to get done. And the concept of keeping the two sets of domains both separate (because they have different though overlapping uses) and *outside* of application code (so that they may be updated without updating the applications themselves) are both appealing .
Jeff C.
Don't harm innocent bystanders.
Discuss mailing list Discuss@lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss
Paul Shupak track@plectere.com
P.S. There was a citation, I believe on NANOG, a couple of weeks ago of another maintained list of multi-level TLDs for IANA listed registries - I'll try to dig it up and send it to you off-list (maybe you can compare it to the current SURBL list).
Hi
2005/11/16, List Mail User track-at-plectere.com |surbl list| <...>:
...
JeffC of SURBL asked:It has been suggested that we could deal with the tripod.com subdomains by adding tripod.com to our two-level-tld list or some equivalent file. Currently the two-level-tld list is hard coded into applications to indicate that domains like co.uk should be checked at the third level, like checkmehere.co.uk. This is somewhat of a kludge, but it works.
How does anyone feel about extending that to tripod.com and potentially other hosts that provide subdomain hosting?
If so, should it be done in the same two-level-tld file or maybe a different, separate file? (Currently the two-level-tld file is mostly geographic domains, i.e. cctlds plus the local country registrar top level domains.) Or should it be done with another, separate DNSBL, etc. Both approaches have advantages and disadvantages. The file is much simpler to maintain for us, but a bit kludgey.
The hard-coding into applications is just plain wrong for a fewreasons; Any list *should* be updatable. Some RHS BLs already support arbitrary subdomains (example: biz.mail.mud.yahoo.com has many more problems than yahoo.com or most yahoo.com subdomains). For DSN, RCVD and other header rules, the "top" domain, (e.g. "ca.us") may not have, or even be required to have an abuse and/or postmaster account (just one possible type of example), but the subdomains either do have them or would trigger tests or rules for them (a domain with no 'A' or 'MX' records may not need to provide any mail services, though its subdomains should and do - your losangeles.ca.us vs. "ca.us" example). Also note that some MTAs (eg. Postfix) already check *all* of the subdomains in a hostname against RHS RBLs.
One problem with this is the syntax for DNS "wildcards" transcendssubdomain levels, so I believe the files/domains should be kept separate from the "true" two-level TLDs, because unfortunately a check for '*.domain.tld' should really be performed also (e.g. '*.domain.tld' in a DNS zone file will match 'l5.l4.l3.domain.tld' as well as the simple 'l3.domain.tld') and this leads to increased overhead for DNS based net tests (without the check, the containing domain cannot be determined by heuristics alone).
I don't fully "get" you're text, but are wildcards not an easy solution?
If a domain is blacklisted it seems normal to blacklists all subdomains. So blacklist *.domain.tld
If only a subdomain is blacklisted add *.subdomain.domain.tld but not *.domain.tld
*.domain.tld can be whitelisted, or it can be a central "surbl" choice only to work with more levels for some domains.
Applications will benefit if they are rewritten to check the full domain instead of a stripped domain. As far as I understand it those written for the current standard will keep working like they are doing now.
The nice part is that the applications don't need to store extra data.
It seems so easy to me that I'm certain that I'm missing something (stupid). I do understand that it's possible there will be less caching from dns lookups. I don't know if wildcards are cached on the wildcard level or with the exact supplied dns name.
I hope this is clear and not to stupid.
Alain
One set of candidates that would also fall into this group includeESPs like cheetah/chtah (discussed a few months ago), where most all (or at least many) of the subdomains are well behaved, but some are definitely "mainsleaze-like" or just plain spammy (not meant to reflect on cheetah specifically here). Similarly, some poorly behaved companies are kind enough to segregate their marketing "spew" into a subdomain, allowing just that to be listed *and* tested for (think of more than just the SURBL lists - but anything SA or other apps may want to check).
Conceivably the file or RBL could be used for any arbitrary number of levels, for example if subdomains like spammerz.losangeles.ca.us (unlikely example, but you get the idea) or something similar started appearing in spams.
The use of most geographic domains in the body of spam is stillrare, but I know that you have seen examples of them used for both dropboxes and domain contacts (and some geographic domains, at least in the ".us" TLD have been suspended for just this cause - directly or indirectly).
If we make the change, we'd need to let the SURBL application authors know to update their tld file, etc., and we'd also need to update our data-side processing to allow subdomains to be listed.
Comments?
The data processing changes needed for applications are the hardestones to get done. And the concept of keeping the two sets of domains both separate (because they have different though overlapping uses) and *outside* of application code (so that they may be updated without updating the applications themselves) are both appealing .
Jeff C.
Don't harm innocent bystanders.
Discuss mailing list Discuss@lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss
Paul Shupak track@plectere.comP.S. There was a citation, I believe on NANOG, a couple of weeks ago of another maintained list of multi-level TLDs for IANA listed registries - I'll try to dig it up and send it to you off-list (maybe you can compare it to the current SURBL list). _______________________________________________ Discuss mailing list Discuss@lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss
-
Alain -
List Mail User